Business Problem:

A group of customers were given an offer in person that they can get a loan at discounted rate and processing fee will be waived off. A pilot campaign was conducted to get response from customers whether they are interested in taking out a loan or not. Response was recorded and data was collected.

Need to Do

  • [x] Build a model to predict whether customers will be interested in taking out a loan or not.
  • [x] Identifying features which are most important
  • [ ] In case of black box models e.g. Random forest use SHAP, LIME to figure out features affecting the target variable
  • [x] Try Unsupervised clustering models
  • [ ] Generate synthetic data for model.
  • [x] Approaching a customer has costs involved with it, hence find the profitable segments so that more customized marketing can be done.
  • [ ] Need to write inferences what is going on
  • [x] Bucketing Age and SCR

Variables involved: Customer_id, Age, Gender, Balance, Occupation, No of Credit transaction, SCR, Holding period

Bucketed
Final visualization
age_d == 1,2,3,4 vs Target, SCR , _Balance

Understanding Variables

  • Holding Period (How long the customer is able to hold the money in his account.. So, if they have some existing expenses like a loan EMI or any other monthly expense which gets deducted, usually the first week of every month, hence it makes the balance in the account lower during initial days of the month itself.Higher the holding period, more stable their money is in the account.)

  • SCR SCR is a score given to a customer for a particular product ( in this case loan ) based on certain parameters, to know whether how likely that customer is to buy that product.. so, higher the score, higher the probability, the customer will buy it.. ###SCR propensity of a customer to respond to a digital marketing

Changes v6:
  1. Now All Models measure recall on same testing data

  2. Fixed Sampling mistake

  3. Redefined print_classification_report as classification_report for better clarity and ease of use

  4. Visualized Decision Trees

  5. Implemented SVC

  6. Implemented KNN which provided great results with default parameters

Changes v7:
  1. Fit Random Forest Models

  2. Fit XgBoost Models

Changes v9:
  1. Added cross validation

  2. Added ROC plots

In [1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns

from sklearn.model_selection import train_test_split, cross_val_score, cross_validate
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.metrics import accuracy_score, recall_score, f1_score, precision_score, roc_curve, auc, plot_roc_curve
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import plot_partial_dependence
from imblearn.under_sampling import NearMiss
from imblearn.over_sampling import SMOTE

import xgboost as xgb
In [2]:
#from pandasgui import show
In [3]:
data = pd.read_csv('Model_data.csv')
#show(data)
In [4]:
data.Balance = data.Balance.astype('int32') #Truncating decimals
In [5]:
data.head()
Out[5]:
Target Age Gender Balance Occupation No_OF_CR_TXNS SCR Holding_Period
0 0 41 M 91519 SELF-EMP 38 926 15
1 0 52 M 117288 SAL 17 768 13
2 0 31 F 259827 SENP 8 816 5
3 0 45 F 26677 PROF 14 353 18
4 0 39 F 43440 SENP 1 751 31
In [6]:
data.Balance.describe()
Out[6]:
count    2.000000e+04
mean     1.461808e+05
std      1.698125e+05
min      0.000000e+00
25%      2.373650e+04
50%      7.975550e+04
75%      2.173100e+05
max      1.246966e+06
Name: Balance, dtype: float64
In [7]:
data.shape
Out[7]:
(20000, 8)
In [8]:
data.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 20000 entries, 0 to 19999
Data columns (total 8 columns):
 #   Column          Non-Null Count  Dtype 
---  ------          --------------  ----- 
 0   Target          20000 non-null  int64 
 1   Age             20000 non-null  int64 
 2   Gender          20000 non-null  object
 3   Balance         20000 non-null  int32 
 4   Occupation      20000 non-null  object
 5   No_OF_CR_TXNS   20000 non-null  int64 
 6   SCR             20000 non-null  int64 
 7   Holding_Period  20000 non-null  int64 
dtypes: int32(1), int64(5), object(2)
memory usage: 1.1+ MB

Gender and Occupation are categorical varibles stored as object type

EDA

No Strong correlations measured except for mild ones in Holding_period and other variables

In [9]:
sns.heatmap(data.corr(), annot=True, square=True) # No strong correlations seen overall
plt.show()
In [10]:
sns.pairplot(data, hue='Target')
plt.show()
In [11]:
sns.countplot(x = data.Gender)
plt.show()
In [12]:
data.Gender.unique()
Out[12]:
array(['M', 'F', 'O'], dtype=object)
In [13]:
data.Occupation.unique()
Out[13]:
array(['SELF-EMP', 'SAL', 'SENP', 'PROF'], dtype=object)
In [14]:
data.Gender.value_counts()
Out[14]:
M    14279
F     5525
O      196
Name: Gender, dtype: int64
In [15]:
data.drop(data.Gender[data.Gender== 'O'].index, axis = 0, inplace= True) # Removed 196 rows with `Gender` = 'O'
In [16]:
data.shape
Out[16]:
(19804, 8)
In [17]:
data.Balance.describe()
Out[17]:
count    1.980400e+04
mean     1.465609e+05
std      1.694485e+05
min      0.000000e+00
25%      2.399900e+04
50%      8.035100e+04
75%      2.183090e+05
max      1.246966e+06
Name: Balance, dtype: float64
In [18]:
# sns.histplot(data.Age)
In [19]:
sns.countplot(x = data.Occupation)
Out[19]:
<matplotlib.axes._subplots.AxesSubplot at 0x1204e1821d0>
In [20]:
sns.countplot(x=data.Target, hue=data.Occupation) ## Self employed are much more likely to take loans
Out[20]:
<matplotlib.axes._subplots.AxesSubplot at 0x1204e5949e8>
In [21]:
g = sns.FacetGrid(data, col='Occupation', hue="Gender")
plt.grid(True)
g.map(sns.countplot, "Gender", alpha=1)
g.add_legend()
plt.grid((False))
C:\Anaconda\lib\site-packages\seaborn\axisgrid.py:723: UserWarning: Using the countplot function without specifying `order` is likely to produce an incorrect plot.
  warnings.warn(warning)

In [22]:
# sns.histplot(data.No_OF_CR_TXNS)
In [23]:
data.No_OF_CR_TXNS.describe()
Out[23]:
count    19804.000000
mean        16.563169
std         12.928214
min          0.000000
25%          7.000000
50%         13.000000
75%         21.000000
max         50.000000
Name: No_OF_CR_TXNS, dtype: float64
In [24]:
sns.violinplot(x=data.No_OF_CR_TXNS)
plt.grid(True)
In [25]:
# len(data[data.No_OF_CR_TXNS==0].index)
In [26]:
# data.drop(index=data[data.No_OF_CR_TXNS==0].index, axis=0)

In [27]:
# sns.displot(data.SCR, kind = 'kde')
sns.distplot(data.SCR)
plt.show()
In [28]:
data.SCR.describe()
Out[28]:
count    19804.000000
mean       557.115987
std        260.562729
min        100.000000
25%        332.000000
50%        560.500000
75%        784.250000
max        999.000000
Name: SCR, dtype: float64
In [29]:
# sns.histplot(data.Holding_Period)
In [30]:
data3 = data
End of Exploratory Data Analysis


Create a function for easy report printing

In [31]:
# A class for pretty printing
class color:
    PURPLE = '\033[95m'
    CYAN = '\033[96m'
    DARKCYAN = '\033[36m'
    BLUE = '\033[94m'
    GREEN = '\033[92m'
    YELLOW = '\033[93m'
    RED = '\033[91m'
    BOLD = '\033[1m'
    UNDERLINE = '\033[4m'
    END = '\033[0m'
    
# function for validation on test data   
def classification_report(y_true, y_prediction, type_of_data='Enter Over/Under/Original sampled', type_of_classifier='ClassifierName'):
    """Print Classification report"""
    
    accuracy = accuracy_score(y_true, y_prediction)
    precision = precision_score(y_true, y_prediction)
    recall = recall_score(y_true, y_prediction)
    f1 = f1_score(y_true, y_prediction)
    
    print('Classification Report on Testing Data:\n'+ color.BOLD + type_of_data, 'data\n'+color.END+color.RED+color.BOLD+type_of_classifier,'Classifier'+color.END+color.END)
    print()
    print('---------------------------------------')
    print(color.BOLD + 'Recall: %s' %recall + color.END)
    print('Precision: %s' %precision)
    print('F1 score: %s' %f1)
    print('Accuracy: %s' %accuracy)
    print('---------------------------------------')
    print()


# A function for cross-validation report    
def cross_val_report(classifier, train_data, train_label, cv=10, scoring=['recall','precision', 'f1','accuracy']):
    
    score = cross_validate(classifier, train_data, train_label, cv=cv, scoring= scoring)
    recall = np.mean(score['test_recall'])
    precision = np.mean(score['test_precision'])
    f1 = np.mean(score['test_f1'])
    accuracy= np.mean(score['test_accuracy'])
    print('Cross Validation Report')
    print(color.BOLD + 'Recall: %s' %recall + color.END)
    print('Precision: %s' %precision)
    print('F1: %s' %f1)
    print('Accuracy: %s' %accuracy)
    print()
    print("*Mean values presented")
    print('---------------------------------------')

Create the first set of training and test data on imbalanced data

In [32]:
df = pd.get_dummies(data, columns=['Gender','Occupation'], drop_first = True)
df.head()
Out[32]:
Target Age Balance No_OF_CR_TXNS SCR Holding_Period Gender_M Occupation_SAL Occupation_SELF-EMP Occupation_SENP
0 0 41 91519 38 926 15 1 0 1 0
1 0 52 117288 17 768 13 1 1 0 0
2 0 31 259827 8 816 5 0 0 0 1
3 0 45 26677 14 353 18 0 0 0 0
4 0 39 43440 1 751 31 0 0 0 1

Creating a model with Original Unbalanced data and measuring metrics

In [33]:
X_original = df.iloc[:,1:]
y_original = df.iloc[:,0]
In [34]:
X_original
Out[34]:
Age Balance No_OF_CR_TXNS SCR Holding_Period Gender_M Occupation_SAL Occupation_SELF-EMP Occupation_SENP
0 41 91519 38 926 15 1 0 1 0
1 52 117288 17 768 13 1 1 0 0
2 31 259827 8 816 5 0 0 0 1
3 45 26677 14 353 18 0 0 0 0
4 39 43440 1 751 31 0 0 0 1
... ... ... ... ... ... ... ... ... ...
19995 23 67612 20 164 27 0 0 0 1
19996 34 472365 8 124 1 0 0 0 0
19997 42 100642 4 263 1 1 1 0 0
19998 36 29950 10 345 31 1 0 0 1
19999 33 48964 4 800 13 1 0 0 0

19804 rows × 9 columns

In [35]:
X_train_orig, X_test_orig, y_train_orig, y_test_orig = train_test_split(X_original,y_original, shuffle = ['True'], stratify=y_original)
In [36]:
clf = DecisionTreeClassifier(max_depth = 5)
clf.fit(X_train_orig, y_train_orig)
y_prediction_orig = clf.predict(X_test_orig)
classification_report(y_test_orig, y_prediction_orig, 'Original', 'Decision Tree')
plot_roc_curve(clf, X_test_orig, y_test_orig)
plt.show()
# cross_val_report(clf, y_test_orig,y_under_prediction.reshape(1,-1))
Classification Report on Testing Data:
Original data
Decision Tree Classifier

---------------------------------------
Recall: 0.04918032786885246
Precision: 0.42
F1 score: 0.0880503144654088
Accuracy: 0.9121389618258937
---------------------------------------

In [37]:
from sklearn.metrics import confusion_matrix
In [38]:
confusion_matrix(y_test_orig, y_prediction_orig)
Out[38]:
array([[4495,   29],
       [ 406,   21]], dtype=int64)
In [39]:
# fig = plt.figure(figsize=(50,20))
# _ = plot_tree(clf, 
#                    feature_names=list(X_original.columns),  
#                    class_names=['0','1'],
#                    filled=True, fontsize=10)

Create undersampled data and fit a model

In [40]:
X_under_train, y_under_train = NearMiss().fit_resample(X_train_orig, y_train_orig)
In [41]:
data[data.Target==1].shape
Out[41]:
(1709, 8)
In [42]:
X_under_train.shape, y_under_train.shape
Out[42]:
((2564, 9), (2564,))
In [43]:
clf_under_sampled = DecisionTreeClassifier(max_depth = 5)
clf_under_sampled.fit(X_under_train, y_under_train)
y_under_prediction = clf_under_sampled.predict(X_test_orig)
classification_report(y_test_orig,y_under_prediction, 'Undersampled', 'Decision Tree')
plot_roc_curve(clf_under_sampled, X_test_orig, y_test_orig)
Classification Report on Testing Data:
Undersampled data
Decision Tree Classifier

---------------------------------------
Recall: 0.7540983606557377
Precision: 0.09168564920273349
F1 score: 0.1634932724041635
Accuracy: 0.3344778832559079
---------------------------------------

Out[43]:
<sklearn.metrics._plot.roc_curve.RocCurveDisplay at 0x1204c18e358>
In [44]:
# cross_val_report(clf_under_sampled, y_test_orig,y_under_prediction)

## crossval here causes unbalanced split
In [45]:
# fig = plt.figure(figsize=(100,100))
# _ = plot_tree(clf_under_sampled, 
#                    feature_names=list(X_original.columns),  
#                    class_names=['0','1'],
#                    filled=True, fontsize=10)

Model on an oversampled dataset

In [46]:
X_over_train, y_over_train = SMOTE().fit_resample(X_original, y_original)
In [47]:
clf_over_sampled = DecisionTreeClassifier(max_depth = 5)
clf_over_sampled.fit(X_over_train, y_over_train)
y_over_predict = clf_over_sampled.predict(X_test_orig)
classification_report(y_test_orig, y_over_predict, 'Oversampled', 'Decision Tree')
plot_roc_curve(clf_over_sampled, X_test_orig, y_test_orig)
Classification Report on Testing Data:
Oversampled data
Decision Tree Classifier

---------------------------------------
Recall: 0.6135831381733021
Precision: 0.1767881241565452
F1 score: 0.2744892613933997
Accuracy: 0.7202585336295698
---------------------------------------

Out[47]:
<sklearn.metrics._plot.roc_curve.RocCurveDisplay at 0x1204cbba828>
In [48]:
# fig = plt.figure(figsize=(100,100))
# _ = plot_tree(clf_over_sampled, 
#                    feature_names=list(X_original.columns),  
#                    class_names=['0','1'],
#                    filled=True, fontsize=10)

In [49]:
print("Original:     "+color.BOLD+ "X_original,y_original"+color.END+"::  X_train_orig, X_test_orig, y_train_orig, y_test_orig")
print()
print("Undersampled:"+color.BOLD+ " X_under, y_under"+color.END+"     ::  X_under_train, y_under_train")
print()
print("Oversampled:"+color.BOLD+ "  X_over, y_over"+color.END+"       ::  X_over_train, y_over_train")
Original:     X_original,y_original::  X_train_orig, X_test_orig, y_train_orig, y_test_orig

Undersampled: X_under, y_under     ::  X_under_train, y_under_train

Oversampled:  X_over, y_over       ::  X_over_train, y_over_train

The above datasets can be better sampled by adjusting hyper-parameters of NearMiss and SMOTE, or other methods of sampling could be used



SVM Classifiers applied

SVC fails to fit on original dataset, possibly because of unbalance

In [50]:
clf_svc0 = SVC()
clf_svc0.fit(X_under_train, y_under_train)
y_predict = clf_svc0.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Undersampled', 'SVM')
plot_roc_curve(clf_svc0, X_test_orig, y_test_orig)
cross_val_report(clf_svc0, X_under_train, y_under_train)
Classification Report on Testing Data:
Undersampled data
SVM Classifier

---------------------------------------
Recall: 0.550351288056206
Precision: 0.07677229663508657
F1 score: 0.13474770642201833
Accuracy: 0.39042617652999395
---------------------------------------

Cross Validation Report
Recall: 0.5155583817829458
Precision: 0.856762028677245
F1: 0.6424574784108211
Accuracy: 0.7136992339494164

*Mean values presented
---------------------------------------
In [51]:
# %%time
# # Will take LONG Time for Training
# clf_svc1 = SVC()
# clf_svc1.fit(X_over_train, y_over_train)
# y_predict = clf_svc1.predict(X_test_orig)
# classification_report(y_test_orig, y_predict, 'Oversampled', 'SVM')
# plot_roc_curve(clf_svc1, X_test_orig, y_test_orig)
# cross_val_report(clf_svc1, X_under_train, y_under_train)


In [52]:
clf_KNN0 = KNeighborsClassifier()
clf_KNN0.fit(X_train_orig, y_train_orig)
y_predict= clf_KNN0.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Original', 'KNN')
plot_roc_curve(clf_KNN0, X_test_orig, y_test_orig)
cross_val_report(clf_KNN0, X_under_train, y_under_train)
Classification Report on Testing Data:
Original data
KNN Classifier

---------------------------------------
Recall: 0.06323185011709602
Precision: 0.23893805309734514
F1 score: 0.1
Accuracy: 0.9018380125227227
---------------------------------------

Cross Validation Report
Recall: 0.6021681201550388
Precision: 0.7188602241504229
F1: 0.650485908766395
Accuracy: 0.6743737840466926

*Mean values presented
---------------------------------------

In [53]:
clf_KNN1 = KNeighborsClassifier()
clf_KNN1.fit(X_under_train, y_under_train)
y_predict= clf_KNN1.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Undersampled', 'KNN')
plot_roc_curve(clf_KNN1, X_test_orig, y_test_orig)
cross_val_report(clf_KNN1, X_under_train, y_under_train)
Classification Report on Testing Data:
Undersampled data
KNN Classifier

---------------------------------------
Recall: 0.7142857142857143
Precision: 0.078125
F1 score: 0.14084507042253522
Accuracy: 0.2484346596647142
---------------------------------------

Cross Validation Report
Recall: 0.6021681201550388
Precision: 0.7188602241504229
F1: 0.650485908766395
Accuracy: 0.6743737840466926

*Mean values presented
---------------------------------------


Random Forest Classifier Models

In [54]:
clf_rf0 = RandomForestClassifier()
clf_rf0.fit(X_train_orig, y_train_orig)
y_predict= clf_rf0.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Original', 'Random Forest')
plot_roc_curve(clf_rf0, X_test_orig, y_test_orig)
cross_val_report(clf_rf0, X_under_train, y_under_train)
Classification Report on Testing Data:
Original data
Random Forest Classifier

---------------------------------------
Recall: 0.11475409836065574
Precision: 0.6282051282051282
F1 score: 0.19405940594059407
Accuracy: 0.9177943849727328
---------------------------------------

Cross Validation Report
Recall: 0.7909883720930233
Precision: 0.7896560936952748
F1: 0.7891872997845422
Accuracy: 0.7886065175097275

*Mean values presented
---------------------------------------
In [55]:
plot_partial_dependence(clf_rf0, X_test_orig, X_test_orig.columns)
Out[55]:
<sklearn.inspection._plot.partial_dependence.PartialDependenceDisplay at 0x1204c1fdc88>

In [56]:
(pd.Series(clf_rf0.feature_importances_, index=X_test_orig.columns)
   .nlargest(4)
   .plot(kind='bar'))
Out[56]:
<matplotlib.axes._subplots.AxesSubplot at 0x1204e6478d0>

RFE Trial

In [57]:
from sklearn.feature_selection import RFE
In [58]:
rfe = RFE(RandomForestClassifier(), n_features_to_select=4)
rfe = rfe.fit(X_over_train, y_over_train)
# summarize the selection of the attributes
print(rfe.support_)
print(rfe.ranking_)
[False  True  True  True  True False False False False]
[2 1 1 1 1 5 3 6 4]
In [59]:
categorical_features_indices = np.where(X_train_orig.dtypes != np.float)[0]
In [60]:
#importing library and building model
from catboost import CatBoostRegressor
model=CatBoostRegressor(iterations=50, depth=3, learning_rate=0.1, loss_function='RMSE')
model.fit(X_train_orig, y_train_orig,cat_features=categorical_features_indices,eval_set=(X_test_orig, y_test_orig),plot=True)
0:	learn: 0.2746514	test: 0.2700960	best: 0.2700960 (0)	total: 150ms	remaining: 7.35s
1:	learn: 0.2695106	test: 0.2611647	best: 0.2611647 (1)	total: 154ms	remaining: 3.7s
2:	learn: 0.2651414	test: 0.2535639	best: 0.2535639 (2)	total: 162ms	remaining: 2.53s
3:	learn: 0.2615590	test: 0.2473106	best: 0.2473106 (3)	total: 165ms	remaining: 1.9s
4:	learn: 0.2584076	test: 0.2429018	best: 0.2429018 (4)	total: 169ms	remaining: 1.52s
5:	learn: 0.2559668	test: 0.2385712	best: 0.2385712 (5)	total: 173ms	remaining: 1.27s
6:	learn: 0.2537854	test: 0.2356387	best: 0.2356387 (6)	total: 180ms	remaining: 1.11s
7:	learn: 0.2520764	test: 0.2325478	best: 0.2325478 (7)	total: 184ms	remaining: 966ms
8:	learn: 0.2506499	test: 0.2299154	best: 0.2299154 (8)	total: 188ms	remaining: 855ms
9:	learn: 0.2492573	test: 0.2281399	best: 0.2281399 (9)	total: 192ms	remaining: 767ms
10:	learn: 0.2481891	test: 0.2261043	best: 0.2261043 (10)	total: 195ms	remaining: 693ms
11:	learn: 0.2472053	test: 0.2249049	best: 0.2249049 (11)	total: 199ms	remaining: 630ms
12:	learn: 0.2464391	test: 0.2235998	best: 0.2235998 (12)	total: 203ms	remaining: 577ms
13:	learn: 0.2458411	test: 0.2227111	best: 0.2227111 (13)	total: 206ms	remaining: 530ms
14:	learn: 0.2454304	test: 0.2218044	best: 0.2218044 (14)	total: 210ms	remaining: 489ms
15:	learn: 0.2450369	test: 0.2211129	best: 0.2211129 (15)	total: 213ms	remaining: 454ms
16:	learn: 0.2445260	test: 0.2204984	best: 0.2204984 (16)	total: 217ms	remaining: 421ms
17:	learn: 0.2440766	test: 0.2201342	best: 0.2201342 (17)	total: 221ms	remaining: 392ms
18:	learn: 0.2437545	test: 0.2196115	best: 0.2196115 (18)	total: 224ms	remaining: 366ms
19:	learn: 0.2433652	test: 0.2192434	best: 0.2192434 (19)	total: 228ms	remaining: 342ms
20:	learn: 0.2429967	test: 0.2189399	best: 0.2189399 (20)	total: 231ms	remaining: 320ms
21:	learn: 0.2426929	test: 0.2187678	best: 0.2187678 (21)	total: 235ms	remaining: 299ms
22:	learn: 0.2423808	test: 0.2185537	best: 0.2185537 (22)	total: 239ms	remaining: 281ms
23:	learn: 0.2419828	test: 0.2184014	best: 0.2184014 (23)	total: 242ms	remaining: 263ms
24:	learn: 0.2417702	test: 0.2181660	best: 0.2181660 (24)	total: 246ms	remaining: 246ms
25:	learn: 0.2415776	test: 0.2179551	best: 0.2179551 (25)	total: 249ms	remaining: 230ms
26:	learn: 0.2413942	test: 0.2178641	best: 0.2178641 (26)	total: 253ms	remaining: 215ms
27:	learn: 0.2413367	test: 0.2177131	best: 0.2177131 (27)	total: 256ms	remaining: 201ms
28:	learn: 0.2412517	test: 0.2175782	best: 0.2175782 (28)	total: 260ms	remaining: 188ms
29:	learn: 0.2410912	test: 0.2174201	best: 0.2174201 (29)	total: 263ms	remaining: 175ms
30:	learn: 0.2409207	test: 0.2173597	best: 0.2173597 (30)	total: 267ms	remaining: 164ms
31:	learn: 0.2408815	test: 0.2173128	best: 0.2173128 (31)	total: 271ms	remaining: 152ms
32:	learn: 0.2408476	test: 0.2172411	best: 0.2172411 (32)	total: 275ms	remaining: 142ms
33:	learn: 0.2407364	test: 0.2171151	best: 0.2171151 (33)	total: 278ms	remaining: 131ms
34:	learn: 0.2406324	test: 0.2170864	best: 0.2170864 (34)	total: 282ms	remaining: 121ms
35:	learn: 0.2404933	test: 0.2170141	best: 0.2170141 (35)	total: 285ms	remaining: 111ms
36:	learn: 0.2404630	test: 0.2169734	best: 0.2169734 (36)	total: 289ms	remaining: 102ms
37:	learn: 0.2403669	test: 0.2169160	best: 0.2169160 (37)	total: 292ms	remaining: 92.3ms
38:	learn: 0.2401428	test: 0.2168655	best: 0.2168655 (38)	total: 296ms	remaining: 83.5ms
39:	learn: 0.2401333	test: 0.2168756	best: 0.2168655 (38)	total: 299ms	remaining: 74.8ms
40:	learn: 0.2400264	test: 0.2167450	best: 0.2167450 (40)	total: 303ms	remaining: 66.5ms
41:	learn: 0.2399334	test: 0.2166843	best: 0.2166843 (41)	total: 306ms	remaining: 58.3ms
42:	learn: 0.2398592	test: 0.2166381	best: 0.2166381 (42)	total: 310ms	remaining: 50.4ms
43:	learn: 0.2397889	test: 0.2166080	best: 0.2166080 (43)	total: 313ms	remaining: 42.7ms
44:	learn: 0.2397687	test: 0.2165998	best: 0.2165998 (44)	total: 317ms	remaining: 35.2ms
45:	learn: 0.2397647	test: 0.2165884	best: 0.2165884 (45)	total: 320ms	remaining: 27.8ms
46:	learn: 0.2397430	test: 0.2165493	best: 0.2165493 (46)	total: 324ms	remaining: 20.7ms
47:	learn: 0.2396924	test: 0.2165821	best: 0.2165493 (46)	total: 328ms	remaining: 13.7ms
48:	learn: 0.2396345	test: 0.2165704	best: 0.2165493 (46)	total: 331ms	remaining: 6.76ms
49:	learn: 0.2396010	test: 0.2165328	best: 0.2165328 (49)	total: 335ms	remaining: 0us

bestTest = 0.2165328422
bestIteration = 49

Out[60]:
<catboost.core.CatBoostRegressor at 0x1204dd089b0>
In [62]:
shap.summary_plot(shap_values, X_test_orig)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-62-43a3214cf379> in <module>
----> 1 shap.summary_plot(shap_values, X_test_orig)

NameError: name 'shap' is not defined
In [63]:
help(CatBoostClassifier)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-63-c2da5a44ed9e> in <module>
----> 1 help(CatBoostClassifier)

NameError: name 'CatBoostClassifier' is not defined
In [64]:
X_train_orig.columns
Out[64]:
Index(['Age', 'Balance', 'No_OF_CR_TXNS', 'SCR', 'Holding_Period', 'Gender_M',
       'Occupation_SAL', 'Occupation_SELF-EMP', 'Occupation_SENP'],
      dtype='object')
In [65]:
clf_rf1 = RandomForestClassifier()
clf_rf1.fit(X_under_train, y_under_train)
y_predict= clf_rf1.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Undersampled', 'Random Forest')
plot_roc_curve(clf_rf1, X_test_orig, y_test_orig)
cross_val_report(clf_rf1, X_under_train, y_under_train)
Classification Report on Testing Data:
Undersampled data
Random Forest Classifier

---------------------------------------
Recall: 0.8430913348946136
Precision: 0.103956107421311
F1 score: 0.18508997429305912
Accuracy: 0.3597253080185821
---------------------------------------

Cross Validation Report
Recall: 0.7917514534883721
Precision: 0.8030155363964994
F1: 0.7966639241192632
Accuracy: 0.7975635335603113

*Mean values presented
---------------------------------------

In [66]:
clf_rf2 = RandomForestClassifier()
In [67]:
clf_rf2.fit(X_under_train, y_under_train)
y_predict= clf_rf2.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Oversampled', 'Random Forest')
plot_roc_curve(clf_rf2, X_test_orig, y_test_orig)
cross_val_report(clf_rf2, X_under_train, y_under_train)
Classification Report on Testing Data:
Oversampled data
Random Forest Classifier

---------------------------------------
Recall: 0.8430913348946136
Precision: 0.10332950631458095
F1 score: 0.18409613909486067
Accuracy: 0.35548374065845284
---------------------------------------

Cross Validation Report
Recall: 0.7862887596899225
Precision: 0.7924655526288069
F1: 0.7887268621085213
Accuracy: 0.788986502918288

*Mean values presented
---------------------------------------
In [68]:
(pd.Series(clf_rf2.feature_importances_, index=X_test_orig.columns)
   .nlargest(4)
   .plot(kind='barh'))
Out[68]:
<matplotlib.axes._subplots.AxesSubplot at 0x1204c552f60>


In [69]:
# rfe = RFE(KNeighborsClassifier(), n_features_to_select=4)
# rfe = rfe.fit(X_train_orig, y_train_orig)
# # summarize the selection of the attributes
# print(rfe.saupport_)
# print(rfe.ranking_)


In [70]:
lr0 = LogisticRegression(max_iter=1000)
lr0.fit(X_train_orig, y_train_orig)
y_predict= lr0.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Original', 'Logistic Regression')
plot_roc_curve(lr0, X_test_orig, y_test_orig)
cross_val_report(lr0, X_under_train, y_under_train)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-70-e967a8300b07> in <module>
----> 1 lr0 = LogisticRegression(max_iter=1000)
      2 lr0.fit(X_train_orig, y_train_orig)
      3 y_predict= lr0.predict(X_test_orig)
      4 classification_report(y_test_orig, y_predict, 'Original', 'Logistic Regression')
      5 plot_roc_curve(lr0, X_test_orig, y_test_orig)

NameError: name 'LogisticRegression' is not defined
In [ ]:
lr1 = LogisticRegression(max_iter=100)
lr1.fit(X_under_train, y_under_train)
y_predict= lr1.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Undersampled', 'Logistic Regression')
plot_roc_curve(lr1, X_test_orig, y_test_orig)
cross_val_report(lr1, X_under_train, y_under_train)
In [ ]:
lr2 = LogisticRegression(max_iter=100)
lr2.fit(X_over_train, y_over_train)
y_predict= lr2.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Oversampled', 'Logistic Regression')
plot_roc_curve(lr1, X_test_orig, y_test_orig)
cross_val_report(lr1, X_under_train, y_under_train)


In [ ]:
clf_KNN2 = KNeighborsClassifier()
clf_KNN2.fit(X_over_train, y_over_train)
y_predict= clf_KNN2.predict(X_test_orig)
classification_report(y_test_orig, y_predict, 'Oversampled', 'KNN')
plot_roc_curve(clf_KNN2, X_test_orig, y_test_orig)
cross_val_report(clf_KNN2, X_under_train, y_under_train)

Segment the customer with prob >90, 80>prob>90, 70>prob>80, 60>prob>70 and so on

In [ ]:
predict_prob = clf_KNN2.predict_proba(X_test_orig)
predict_prob.shape
# print more significant digits here
In [ ]:
# predict_prob
In [ ]:
predictions = pd.DataFrame(predict_prob, columns=[0, 'PredictionProb'])
In [ ]:
# predictions.head()
In [71]:
predictions_prob = predictions.drop(0, axis=1)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-71-790e39311813> in <module>
----> 1 predictions_prob = predictions.drop(0, axis=1)

NameError: name 'predictions' is not defined
In [72]:
y = pd.DataFrame(y_test_orig, columns = ['Target'])
y = y.reset_index(drop=True)
In [73]:
error = y.Target- predictions_prob.PredictionProb
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-73-500109b319be> in <module>
----> 1 error = y.Target- predictions_prob.PredictionProb

NameError: name 'predictions_prob' is not defined
In [74]:
# This series indicate the error, closer to zero values mean better prediction
# positive mean that my model should have predicted loan taker, but did not
# negative means that model should have predicted NOT a loan taker, but did predict as such
In [75]:
# target - probablity
In [76]:
plt.hist(error)
plt.xlabel('Error in Prediction: Target - Probablity Of Prediction')
plt.ylabel("Count")
plt.show()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-76-1dfda7660d6e> in <module>
----> 1 plt.hist(error)
      2 plt.xlabel('Error in Prediction: Target - Probablity Of Prediction')
      3 plt.ylabel("Count")
      4 plt.show()

NameError: name 'error' is not defined

PRE-BUCKETING VISUALIZATIONS FOR INFERENCES

In [77]:
sns.catplot(x="Occupation", y="SCR", hue = "Target",data=data, kind = "violin", split = True)
plt.show()
In [78]:
sns.relplot(x="Occupation", y="SCR", hue = "Target",data=data, aspect=1.5, kind = "line")
plt.show()
In [79]:
sns.catplot(x="Occupation", y="SCR", hue = "Target",data=data, aspect=2.0, kind = "point")
plt.show()
In [80]:
sns.catplot(x="Occupation", y="SCR", hue = "Target",data=data, kind = "violin", split = True)
plt.show()

The model is good, make few mistakes

In [81]:
data1 = data.copy()

Bucketing for AGE values

In [82]:
data1['Ageb'] = pd.qcut(data['Age'], q=4)
In [83]:
data1.head()
Out[83]:
Target Age Gender Balance Occupation No_OF_CR_TXNS SCR Holding_Period Ageb
0 0 41 M 91519 SELF-EMP 38 926 15 (38.0, 46.0]
1 0 52 M 117288 SAL 17 768 13 (46.0, 55.0]
2 0 31 F 259827 SENP 8 816 5 (30.0, 38.0]
3 0 45 F 26677 PROF 14 353 18 (38.0, 46.0]
4 0 39 F 43440 SENP 1 751 31 (38.0, 46.0]
In [84]:
data1.Ageb.value_counts()
Out[84]:
(30.0, 38.0]      5188
(20.999, 30.0]    5136
(46.0, 55.0]      4936
(38.0, 46.0]      4544
Name: Ageb, dtype: int64
In [85]:
data1.dtypes
Out[85]:
Target               int64
Age                  int64
Gender              object
Balance              int32
Occupation          object
No_OF_CR_TXNS        int64
SCR                  int64
Holding_Period       int64
Ageb              category
dtype: object
In [86]:
data1.Ageb
Out[86]:
0          (38.0, 46.0]
1          (46.0, 55.0]
2          (30.0, 38.0]
3          (38.0, 46.0]
4          (38.0, 46.0]
              ...      
19995    (20.999, 30.0]
19996      (30.0, 38.0]
19997      (38.0, 46.0]
19998      (30.0, 38.0]
19999      (30.0, 38.0]
Name: Ageb, Length: 19804, dtype: category
Categories (4, interval[float64]): [(20.999, 30.0] < (30.0, 38.0] < (38.0, 46.0] < (46.0, 55.0]]
In [87]:
data1.head(5)
Out[87]:
Target Age Gender Balance Occupation No_OF_CR_TXNS SCR Holding_Period Ageb
0 0 41 M 91519 SELF-EMP 38 926 15 (38.0, 46.0]
1 0 52 M 117288 SAL 17 768 13 (46.0, 55.0]
2 0 31 F 259827 SENP 8 816 5 (30.0, 38.0]
3 0 45 F 26677 PROF 14 353 18 (38.0, 46.0]
4 0 39 F 43440 SENP 1 751 31 (38.0, 46.0]
In [88]:
# sns.pairplot(data1, hue='Target')
In [89]:
data1 = pd.get_dummies(data1)
In [90]:
data1.head(2)
Out[90]:
Target Age Balance No_OF_CR_TXNS SCR Holding_Period Gender_F Gender_M Occupation_PROF Occupation_SAL Occupation_SELF-EMP Occupation_SENP Ageb_(20.999, 30.0] Ageb_(30.0, 38.0] Ageb_(38.0, 46.0] Ageb_(46.0, 55.0]
0 0 41 91519 38 926 15 0 1 0 0 1 0 0 0 1 0
1 0 52 117288 17 768 13 0 1 0 1 0 0 0 0 0 1
In [91]:
data2 = data1.copy()
In [92]:
data1.columns
Out[92]:
Index(['Target', 'Age', 'Balance', 'No_OF_CR_TXNS', 'SCR', 'Holding_Period',
       'Gender_F', 'Gender_M', 'Occupation_PROF', 'Occupation_SAL',
       'Occupation_SELF-EMP', 'Occupation_SENP', 'Ageb_(20.999, 30.0]',
       'Ageb_(30.0, 38.0]', 'Ageb_(38.0, 46.0]', 'Ageb_(46.0, 55.0]'],
      dtype='object')
In [93]:
cols = ['Ageb_(30.0, 38.0]', 'Ageb_(20.999, 30.0]','Ageb_(46.0, 55.0]','Ageb_(38.0, 46.0]']

def func1(x):
    if x ==1:
        return 1
def func2(x):
    if x ==1:
        return 2

def func3(x):
    if x ==1:
        return 3

def func4(x):
    if x ==1:
        return 4


data1['age_d'] = data1['Ageb_(30.0, 38.0]'].apply(func1)
data1['age_d'] = data1['Ageb_(20.999, 30.0]'].apply(func2)
data1['age_d'] = data1['Ageb_(46.0, 55.0]'].apply(func3)
data1['age_d'] = data1['Ageb_(38.0, 46.0]'].apply(func4)
In [94]:
for ind, row in data1.iterrows():
    if row['Ageb_(20.999, 30.0]'] ==1:
        data1.loc[ind, 'age_d'] = 1
    elif row['Ageb_(30.0, 38.0]'] ==1:
        data1.loc[ind, 'age_d'] = 2
    elif row['Ageb_(38.0, 46.0]'] ==1:
        data1.loc[ind, 'age_d'] = 3
    elif row['Ageb_(46.0, 55.0]'] ==1:
        data1.loc[ind, 'age_d'] = 4
In [95]:
data1.age_d.value_counts()
Out[95]:
2.0    5188
1.0    5136
4.0    4936
3.0    4544
Name: age_d, dtype: int64
In [96]:
data1.age_d = data1.age_d.astype('int32')
In [97]:
data1.head()
Out[97]:
Target Age Balance No_OF_CR_TXNS SCR Holding_Period Gender_F Gender_M Occupation_PROF Occupation_SAL Occupation_SELF-EMP Occupation_SENP Ageb_(20.999, 30.0] Ageb_(30.0, 38.0] Ageb_(38.0, 46.0] Ageb_(46.0, 55.0] age_d
0 0 41 91519 38 926 15 0 1 0 0 1 0 0 0 1 0 3
1 0 52 117288 17 768 13 0 1 0 1 0 0 0 0 0 1 4
2 0 31 259827 8 816 5 1 0 0 0 0 1 0 1 0 0 2
3 0 45 26677 14 353 18 1 0 1 0 0 0 0 0 1 0 3
4 0 39 43440 1 751 31 1 0 0 0 0 1 0 0 1 0 3

POST-BUCKETING VISUALIZATIONS FOR INFERENCES

In [98]:
sns.catplot(x="age_d", y="SCR", hue = "Target",data=data1, kind = "violin", split = True)
plt.show()
In [99]:
data1.columns
Out[99]:
Index(['Target', 'Age', 'Balance', 'No_OF_CR_TXNS', 'SCR', 'Holding_Period',
       'Gender_F', 'Gender_M', 'Occupation_PROF', 'Occupation_SAL',
       'Occupation_SELF-EMP', 'Occupation_SENP', 'Ageb_(20.999, 30.0]',
       'Ageb_(30.0, 38.0]', 'Ageb_(38.0, 46.0]', 'Ageb_(46.0, 55.0]', 'age_d'],
      dtype='object')
In [100]:
sns.scatterplot(x='age_d', y="SCR", hue = "Target",data=data1,)
plt.show()
In [101]:
sns.catplot(x="age_d", y="SCR", hue = "Target",data=data1, kind = "swarm")
plt.show()
In [102]:
sns.set_style('ticks')
In [103]:
g = sns.relplot(x="SCR", y="age_d", hue = "Target",data=data1, aspect=3, kind = "line")
# g.figure.set_size_inches(18.5, 10.5)
# sns.despine()

BUCKETING SCR & HOLDING PERIODS

In [104]:
data1['SCRb'] = pd.qcut(data['SCR'], q=5)
In [105]:
data1.SCRb.value_counts()
Out[105]:
(99.999, 288.0]    3987
(468.0, 651.0]     3964
(826.0, 999.0]     3956
(651.0, 826.0]     3952
(288.0, 468.0]     3945
Name: SCRb, dtype: int64
In [106]:
data1['HPB'] = pd.qcut(data['Holding_Period'], q=4)
In [107]:
data1.HPB.value_counts()
Out[107]:
(0.999, 8.0]    5519
(16.0, 23.0]    4998
(8.0, 16.0]     4791
(23.0, 31.0]    4496
Name: HPB, dtype: int64
In [108]:
data1.head()
Out[108]:
Target Age Balance No_OF_CR_TXNS SCR Holding_Period Gender_F Gender_M Occupation_PROF Occupation_SAL Occupation_SELF-EMP Occupation_SENP Ageb_(20.999, 30.0] Ageb_(30.0, 38.0] Ageb_(38.0, 46.0] Ageb_(46.0, 55.0] age_d SCRb HPB
0 0 41 91519 38 926 15 0 1 0 0 1 0 0 0 1 0 3 (826.0, 999.0] (8.0, 16.0]
1 0 52 117288 17 768 13 0 1 0 1 0 0 0 0 0 1 4 (651.0, 826.0] (8.0, 16.0]
2 0 31 259827 8 816 5 1 0 0 0 0 1 0 1 0 0 2 (651.0, 826.0] (0.999, 8.0]
3 0 45 26677 14 353 18 1 0 1 0 0 0 0 0 1 0 3 (288.0, 468.0] (16.0, 23.0]
4 0 39 43440 1 751 31 1 0 0 0 0 1 0 0 1 0 3 (651.0, 826.0] (23.0, 31.0]
In [109]:
sns.catplot(x="age_d", y="SCRb", hue = "Target",data=data1, kind = "violin", split = True)
plt.show()
In [110]:
sns.catplot(x="age_d", y="HPB", hue = "Target",data=data1, kind = "violin", split = True)
plt.show()
In [116]:
sns.pairplot(data1, hue = 'Target')
plt.show()
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
Default bandwidth for data is 0; skipping density estimation.
In [ ]:
 

SHAP

In [111]:
import shap
In [112]:
X_train_summary = shap.kmeans(X_train_orig, 10)
In [113]:
explainerKNN = shap.KernelExplainer(clf_KNN0.predict,X_train_summary)
In [114]:
shap_values_KNN_test = explainerKNN.shap_values(X_test_orig)

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-114-e69a2a2b50e7> in <module>
----> 1 shap_values_KNN_test = explainerKNN.shap_values(X_test_orig)

C:\Anaconda\lib\site-packages\shap\explainers\_kernel.py in shap_values(self, X, **kwargs)
    181                 if self.keep_index:
    182                     data = convert_to_instance_with_index(data, column_name, index_value[i:i + 1], index_name)
--> 183                 explanations.append(self.explain(data, **kwargs))
    184 
    185             # vector-output

C:\Anaconda\lib\site-packages\shap\explainers\_kernel.py in explain(self, incoming_instance, **kwargs)
    371 
    372             # execute the model on the synthetic samples we have created
--> 373             self.run()
    374 
    375             # solve then expand the feature importance (Shapley value) vector to contain the non-varying features

C:\Anaconda\lib\site-packages\shap\explainers\_kernel.py in run(self)
    508             if self.keep_index_ordered:
    509                 data = data.sort_index()
--> 510         modelOut = self.model.f(data)
    511         if isinstance(modelOut, (pd.DataFrame, pd.Series)):
    512             modelOut = modelOut.values

C:\Anaconda\lib\site-packages\sklearn\neighbors\_classification.py in predict(self, X)
    187         for k, classes_k in enumerate(classes_):
    188             if weights is None:
--> 189                 mode, _ = stats.mode(_y[neigh_ind, k], axis=1)
    190             else:
    191                 mode, _ = weighted_mode(_y[neigh_ind, k], weights, axis=1)

C:\Anaconda\lib\site-packages\scipy\stats\stats.py in mode(a, axis, nan_policy)
    570     counts = np.zeros(a_view.shape[:-1], dtype=np.int)
    571     for ind in inds:
--> 572         modes[ind], counts[ind] = _mode1D(a_view[ind])
    573     newshape = list(a.shape)
    574     newshape[axis] = 1

C:\Anaconda\lib\site-packages\scipy\stats\stats.py in _mode1D(a)
    557 
    558     def _mode1D(a):
--> 559         vals, cnts = np.unique(a, return_counts=True)
    560         return vals[cnts.argmax()], cnts.max()
    561 

<__array_function__ internals> in unique(*args, **kwargs)

C:\Anaconda\lib\site-packages\numpy\lib\arraysetops.py in unique(ar, return_index, return_inverse, return_counts, axis)
    261     ar = np.asanyarray(ar)
    262     if axis is None:
--> 263         ret = _unique1d(ar, return_index, return_inverse, return_counts)
    264         return _unpack_tuple(ret)
    265 

C:\Anaconda\lib\site-packages\numpy\lib\arraysetops.py in _unique1d(ar, return_index, return_inverse, return_counts)
    313     mask = np.empty(aux.shape, dtype=np.bool_)
    314     mask[:1] = True
--> 315     mask[1:] = aux[1:] != aux[:-1]
    316 
    317     ret = (aux[mask],)

KeyboardInterrupt: 
In [ ]:
shap.initjs()
shap.force_plot(explainerKNN.expected_value, shap_values_KNN_test[:1000,:], X_test_orig.iloc[:1000,:])

LIME

In [ ]:
import lime
import lime.lime_tabular
In [ ]:
lgb_params = {
    'task': 'train',
    'boosting_type': 'goss',
    'objective': 'binary',
    'metric':'binary_logloss',
    'metric': {'l2', 'auc'},
    'num_leaves': 50,
    'learning_rate': 0.1,
    'feature_fraction': 0.8,
    'bagging_fraction': 0.8,
    'verbose': None,
    'num_iteration':100,
    'num_threads':7,
    'max_depth':12,
    'min_data_in_leaf':100,
    'alpha':0.5}
In [ ]:
import warnings
from lime import submodular_pick
import lightgbm as lgb

lgb_train = lgb.Dataset(X_train_orig, y_train_orig)
lgb_eval = lgb.Dataset(X_test_orig, y_test_orig)

model = lgb.train(lgb_params,lgb_train,num_boost_round=20,valid_sets=lgb_eval,early_stopping_rounds=5)
lime.lime_tabular.LimeTabularExplainer(data[model.feature_name()].astype(int).values,  
mode='classification',training_labels=data3['Target'],feature_names=model.feature_name())
# Remember to convert the dataframe to matrix values
# SP-LIME returns exaplanations on a sample set to provide a non redundant global decision boundary of original model
sp_obj = submodular_pick.SubmodularPick(explainer,data3[model.feature_name()].values, \
prob, num_features=5,num_exps_desired=10)

[exp.as_pyplot_figure(label=1) for exp in sp_obj.sp_explanations]
In [119]:
from catboost import CatBoostClassifier
model = CatBoostClassifier(iterations=1500, learning_rate=0.01, l2_leaf_reg=3.5, depth=5, rsm=0.98, 
                           loss_function= 'Logloss', eval_metric='AUC',use_best_model=True,random_seed=42)
In [122]:
cate_features_index = np.where(data.dtypes != float)[0]
In [123]:
model.fit(X_train_orig,y_train_orig,cat_features=cate_features_index,eval_set=(X_test_orig,y_test_orig))
0:	test: 0.7088303	best: 0.7088303 (0)	total: 91.9ms	remaining: 2m 17s
1:	test: 0.7088303	best: 0.7088303 (0)	total: 154ms	remaining: 1m 55s
2:	test: 0.7088303	best: 0.7088303 (0)	total: 162ms	remaining: 1m 20s
3:	test: 0.7437683	best: 0.7437683 (3)	total: 189ms	remaining: 1m 10s
4:	test: 0.7490941	best: 0.7490941 (4)	total: 214ms	remaining: 1m 3s
5:	test: 0.7525746	best: 0.7525746 (5)	total: 237ms	remaining: 59s
6:	test: 0.7534930	best: 0.7534930 (6)	total: 261ms	remaining: 55.8s
7:	test: 0.7555706	best: 0.7555706 (7)	total: 272ms	remaining: 50.8s
8:	test: 0.7522242	best: 0.7555706 (7)	total: 281ms	remaining: 46.6s
9:	test: 0.7512243	best: 0.7555706 (7)	total: 326ms	remaining: 48.5s
10:	test: 0.7489704	best: 0.7555706 (7)	total: 336ms	remaining: 45.4s
11:	test: 0.7489704	best: 0.7555706 (7)	total: 342ms	remaining: 42.4s
12:	test: 0.7489704	best: 0.7555706 (7)	total: 349ms	remaining: 39.9s
13:	test: 0.7489704	best: 0.7555706 (7)	total: 356ms	remaining: 37.8s
14:	test: 0.7489714	best: 0.7555706 (7)	total: 366ms	remaining: 36.3s
15:	test: 0.7528741	best: 0.7555706 (7)	total: 376ms	remaining: 34.9s
16:	test: 0.7502060	best: 0.7555706 (7)	total: 387ms	remaining: 33.7s
17:	test: 0.7516376	best: 0.7555706 (7)	total: 396ms	remaining: 32.6s
18:	test: 0.7516376	best: 0.7555706 (7)	total: 402ms	remaining: 31.3s
19:	test: 0.7526727	best: 0.7555706 (7)	total: 422ms	remaining: 31.2s
20:	test: 0.7504219	best: 0.7555706 (7)	total: 432ms	remaining: 30.5s
21:	test: 0.7504219	best: 0.7555706 (7)	total: 438ms	remaining: 29.4s
22:	test: 0.7496438	best: 0.7555706 (7)	total: 447ms	remaining: 28.7s
23:	test: 0.7483005	best: 0.7555706 (7)	total: 465ms	remaining: 28.6s
24:	test: 0.7483005	best: 0.7555706 (7)	total: 472ms	remaining: 27.8s
25:	test: 0.7492235	best: 0.7555706 (7)	total: 498ms	remaining: 28.2s
26:	test: 0.7499899	best: 0.7555706 (7)	total: 540ms	remaining: 29.5s
27:	test: 0.7518043	best: 0.7555706 (7)	total: 550ms	remaining: 28.9s
28:	test: 0.7540437	best: 0.7555706 (7)	total: 559ms	remaining: 28.4s
29:	test: 0.7540437	best: 0.7555706 (7)	total: 568ms	remaining: 27.8s
30:	test: 0.7555118	best: 0.7555706 (7)	total: 578ms	remaining: 27.4s
31:	test: 0.7551340	best: 0.7555706 (7)	total: 585ms	remaining: 26.9s
32:	test: 0.7551340	best: 0.7555706 (7)	total: 592ms	remaining: 26.3s
33:	test: 0.7551340	best: 0.7555706 (7)	total: 602ms	remaining: 26s
34:	test: 0.7546101	best: 0.7555706 (7)	total: 615ms	remaining: 25.7s
35:	test: 0.7543316	best: 0.7555706 (7)	total: 624ms	remaining: 25.4s
36:	test: 0.7533278	best: 0.7555706 (7)	total: 632ms	remaining: 25s
37:	test: 0.7507690	best: 0.7555706 (7)	total: 647ms	remaining: 24.9s
38:	test: 0.7507690	best: 0.7555706 (7)	total: 654ms	remaining: 24.5s
39:	test: 0.7527320	best: 0.7555706 (7)	total: 680ms	remaining: 24.8s
40:	test: 0.7512209	best: 0.7555706 (7)	total: 690ms	remaining: 24.6s
41:	test: 0.7498548	best: 0.7555706 (7)	total: 707ms	remaining: 24.5s
42:	test: 0.7511547	best: 0.7555706 (7)	total: 718ms	remaining: 24.3s
43:	test: 0.7512551	best: 0.7555706 (7)	total: 750ms	remaining: 24.8s
44:	test: 0.7512551	best: 0.7555706 (7)	total: 756ms	remaining: 24.5s
45:	test: 0.7512551	best: 0.7555706 (7)	total: 762ms	remaining: 24.1s
46:	test: 0.7798090	best: 0.7798090 (46)	total: 784ms	remaining: 24.2s
47:	test: 0.7945413	best: 0.7945413 (47)	total: 799ms	remaining: 24.2s
48:	test: 0.8075761	best: 0.8075761 (48)	total: 827ms	remaining: 24.5s
49:	test: 0.8165229	best: 0.8165229 (49)	total: 845ms	remaining: 24.5s
50:	test: 0.8213584	best: 0.8213584 (50)	total: 871ms	remaining: 24.7s
51:	test: 0.8246777	best: 0.8246777 (51)	total: 882ms	remaining: 24.6s
52:	test: 0.8278714	best: 0.8278714 (52)	total: 898ms	remaining: 24.5s
53:	test: 0.8316002	best: 0.8316002 (53)	total: 930ms	remaining: 24.9s
54:	test: 0.8333603	best: 0.8333603 (54)	total: 950ms	remaining: 24.9s
55:	test: 0.8348387	best: 0.8348387 (55)	total: 967ms	remaining: 24.9s
56:	test: 0.8358445	best: 0.8358445 (56)	total: 983ms	remaining: 24.9s
57:	test: 0.8369026	best: 0.8369026 (57)	total: 1.01s	remaining: 25.2s
58:	test: 0.8375378	best: 0.8375378 (58)	total: 1.02s	remaining: 25.1s
59:	test: 0.8378650	best: 0.8378650 (59)	total: 1.06s	remaining: 25.4s
60:	test: 0.8381971	best: 0.8381971 (60)	total: 1.09s	remaining: 25.7s
61:	test: 0.8387114	best: 0.8387114 (61)	total: 1.11s	remaining: 25.7s
62:	test: 0.8385814	best: 0.8387114 (61)	total: 1.14s	remaining: 25.9s
63:	test: 0.8384556	best: 0.8387114 (61)	total: 1.15s	remaining: 25.7s
64:	test: 0.8391056	best: 0.8391056 (64)	total: 1.16s	remaining: 25.5s
65:	test: 0.8391874	best: 0.8391874 (65)	total: 1.17s	remaining: 25.4s
66:	test: 0.8393038	best: 0.8393038 (66)	total: 1.17s	remaining: 25.1s
67:	test: 0.8394993	best: 0.8394993 (67)	total: 1.18s	remaining: 24.9s
68:	test: 0.8393942	best: 0.8394993 (67)	total: 1.2s	remaining: 24.8s
69:	test: 0.8397102	best: 0.8397102 (69)	total: 1.21s	remaining: 24.7s
70:	test: 0.8397216	best: 0.8397216 (70)	total: 1.22s	remaining: 24.6s
71:	test: 0.8393825	best: 0.8397216 (70)	total: 1.24s	remaining: 24.6s
72:	test: 0.8394115	best: 0.8397216 (70)	total: 1.25s	remaining: 24.5s
73:	test: 0.8391084	best: 0.8397216 (70)	total: 1.27s	remaining: 24.5s
74:	test: 0.8391027	best: 0.8397216 (70)	total: 1.28s	remaining: 24.3s
75:	test: 0.8392358	best: 0.8397216 (70)	total: 1.29s	remaining: 24.1s
76:	test: 0.8393942	best: 0.8397216 (70)	total: 1.3s	remaining: 24s
77:	test: 0.8393900	best: 0.8397216 (70)	total: 1.31s	remaining: 23.8s
78:	test: 0.8393911	best: 0.8397216 (70)	total: 1.33s	remaining: 23.9s
79:	test: 0.8393486	best: 0.8397216 (70)	total: 1.34s	remaining: 23.8s
80:	test: 0.8393616	best: 0.8397216 (70)	total: 1.35s	remaining: 23.6s
81:	test: 0.8396579	best: 0.8397216 (70)	total: 1.36s	remaining: 23.5s
82:	test: 0.8397485	best: 0.8397485 (82)	total: 1.37s	remaining: 23.4s
83:	test: 0.8392422	best: 0.8397485 (82)	total: 1.42s	remaining: 23.9s
84:	test: 0.8389420	best: 0.8397485 (82)	total: 1.44s	remaining: 24s
85:	test: 0.8389932	best: 0.8397485 (82)	total: 1.46s	remaining: 24s
86:	test: 0.8389932	best: 0.8397485 (82)	total: 1.47s	remaining: 23.8s
87:	test: 0.8388304	best: 0.8397485 (82)	total: 1.48s	remaining: 23.8s
88:	test: 0.8395075	best: 0.8397485 (82)	total: 1.5s	remaining: 23.7s
89:	test: 0.8400480	best: 0.8400480 (89)	total: 1.52s	remaining: 23.8s
90:	test: 0.8402949	best: 0.8402949 (90)	total: 1.53s	remaining: 23.8s
91:	test: 0.8403286	best: 0.8403286 (91)	total: 1.56s	remaining: 24s
92:	test: 0.8403159	best: 0.8403286 (91)	total: 1.59s	remaining: 24s
93:	test: 0.8408589	best: 0.8408589 (93)	total: 1.6s	remaining: 24s
94:	test: 0.8408573	best: 0.8408589 (93)	total: 1.61s	remaining: 23.8s
95:	test: 0.8406844	best: 0.8408589 (93)	total: 1.62s	remaining: 23.8s
96:	test: 0.8407331	best: 0.8408589 (93)	total: 1.64s	remaining: 23.7s
97:	test: 0.8409632	best: 0.8409632 (97)	total: 1.65s	remaining: 23.6s
98:	test: 0.8405320	best: 0.8409632 (97)	total: 1.69s	remaining: 24s
99:	test: 0.8405294	best: 0.8409632 (97)	total: 1.7s	remaining: 23.8s
100:	test: 0.8409715	best: 0.8409715 (100)	total: 1.72s	remaining: 23.8s
101:	test: 0.8413502	best: 0.8413502 (101)	total: 1.74s	remaining: 23.8s
102:	test: 0.8416822	best: 0.8416822 (102)	total: 1.75s	remaining: 23.7s
103:	test: 0.8416491	best: 0.8416822 (102)	total: 1.77s	remaining: 23.8s
104:	test: 0.8416481	best: 0.8416822 (102)	total: 1.78s	remaining: 23.7s
105:	test: 0.8416113	best: 0.8416822 (102)	total: 1.8s	remaining: 23.7s
106:	test: 0.8416072	best: 0.8416822 (102)	total: 1.82s	remaining: 23.6s
107:	test: 0.8416165	best: 0.8416822 (102)	total: 1.82s	remaining: 23.5s
108:	test: 0.8416579	best: 0.8416822 (102)	total: 1.84s	remaining: 23.5s
109:	test: 0.8416740	best: 0.8416822 (102)	total: 1.85s	remaining: 23.4s
110:	test: 0.8418303	best: 0.8418303 (110)	total: 1.88s	remaining: 23.5s
111:	test: 0.8416879	best: 0.8418303 (110)	total: 1.89s	remaining: 23.5s
112:	test: 0.8416895	best: 0.8418303 (110)	total: 1.92s	remaining: 23.5s
113:	test: 0.8416828	best: 0.8418303 (110)	total: 1.93s	remaining: 23.4s
114:	test: 0.8420332	best: 0.8420332 (114)	total: 1.95s	remaining: 23.5s
115:	test: 0.8420073	best: 0.8420332 (114)	total: 1.96s	remaining: 23.4s
116:	test: 0.8417977	best: 0.8420332 (114)	total: 1.97s	remaining: 23.3s
117:	test: 0.8418137	best: 0.8420332 (114)	total: 1.99s	remaining: 23.3s
118:	test: 0.8417728	best: 0.8420332 (114)	total: 2.01s	remaining: 23.3s
119:	test: 0.8422698	best: 0.8422698 (119)	total: 2.02s	remaining: 23.3s
120:	test: 0.8423128	best: 0.8423128 (120)	total: 2.04s	remaining: 23.2s
121:	test: 0.8423418	best: 0.8423418 (121)	total: 2.06s	remaining: 23.3s
122:	test: 0.8422315	best: 0.8423418 (121)	total: 2.08s	remaining: 23.3s
123:	test: 0.8422170	best: 0.8423418 (121)	total: 2.09s	remaining: 23.2s
124:	test: 0.8420550	best: 0.8423418 (121)	total: 2.1s	remaining: 23.1s
125:	test: 0.8420451	best: 0.8423418 (121)	total: 2.12s	remaining: 23.1s
126:	test: 0.8420112	best: 0.8423418 (121)	total: 2.13s	remaining: 23s
127:	test: 0.8419750	best: 0.8423418 (121)	total: 2.14s	remaining: 22.9s
128:	test: 0.8418740	best: 0.8423418 (121)	total: 2.16s	remaining: 22.9s
129:	test: 0.8420110	best: 0.8423418 (121)	total: 2.17s	remaining: 22.9s
130:	test: 0.8419043	best: 0.8423418 (121)	total: 2.18s	remaining: 22.8s
131:	test: 0.8418779	best: 0.8423418 (121)	total: 2.2s	remaining: 22.8s
132:	test: 0.8422506	best: 0.8423418 (121)	total: 2.21s	remaining: 22.7s
133:	test: 0.8424463	best: 0.8424463 (133)	total: 2.26s	remaining: 23s
134:	test: 0.8423718	best: 0.8424463 (133)	total: 2.27s	remaining: 23s
135:	test: 0.8421777	best: 0.8424463 (133)	total: 2.29s	remaining: 23s
136:	test: 0.8421761	best: 0.8424463 (133)	total: 2.3s	remaining: 22.9s
137:	test: 0.8421761	best: 0.8424463 (133)	total: 2.31s	remaining: 22.8s
138:	test: 0.8425644	best: 0.8425644 (138)	total: 2.32s	remaining: 22.7s
139:	test: 0.8428926	best: 0.8428926 (139)	total: 2.33s	remaining: 22.7s
140:	test: 0.8430111	best: 0.8430111 (140)	total: 2.35s	remaining: 22.6s
141:	test: 0.8428786	best: 0.8430111 (140)	total: 2.36s	remaining: 22.6s
142:	test: 0.8427212	best: 0.8430111 (140)	total: 2.38s	remaining: 22.5s
143:	test: 0.8429319	best: 0.8430111 (140)	total: 2.41s	remaining: 22.7s
144:	test: 0.8428224	best: 0.8430111 (140)	total: 2.42s	remaining: 22.6s
145:	test: 0.8429384	best: 0.8430111 (140)	total: 2.43s	remaining: 22.5s
146:	test: 0.8429549	best: 0.8430111 (140)	total: 2.45s	remaining: 22.5s
147:	test: 0.8428095	best: 0.8430111 (140)	total: 2.46s	remaining: 22.5s
148:	test: 0.8424675	best: 0.8430111 (140)	total: 2.47s	remaining: 22.4s
149:	test: 0.8422807	best: 0.8430111 (140)	total: 2.49s	remaining: 22.4s
150:	test: 0.8422509	best: 0.8430111 (140)	total: 2.5s	remaining: 22.4s
151:	test: 0.8422530	best: 0.8430111 (140)	total: 2.51s	remaining: 22.3s
152:	test: 0.8424160	best: 0.8430111 (140)	total: 2.56s	remaining: 22.5s
153:	test: 0.8424916	best: 0.8430111 (140)	total: 2.57s	remaining: 22.5s
154:	test: 0.8428162	best: 0.8430111 (140)	total: 2.59s	remaining: 22.5s
155:	test: 0.8430243	best: 0.8430243 (155)	total: 2.6s	remaining: 22.4s
156:	test: 0.8431853	best: 0.8431853 (156)	total: 2.61s	remaining: 22.3s
157:	test: 0.8432743	best: 0.8432743 (157)	total: 2.62s	remaining: 22.3s
158:	test: 0.8435730	best: 0.8435730 (158)	total: 2.63s	remaining: 22.2s
159:	test: 0.8436564	best: 0.8436564 (159)	total: 2.64s	remaining: 22.1s
160:	test: 0.8438712	best: 0.8438712 (160)	total: 2.65s	remaining: 22.1s
161:	test: 0.8439043	best: 0.8439043 (161)	total: 2.67s	remaining: 22.1s
162:	test: 0.8438764	best: 0.8439043 (161)	total: 2.69s	remaining: 22s
163:	test: 0.8437648	best: 0.8439043 (161)	total: 2.7s	remaining: 22s
164:	test: 0.8438332	best: 0.8439043 (161)	total: 2.71s	remaining: 21.9s
165:	test: 0.8439434	best: 0.8439434 (165)	total: 2.72s	remaining: 21.9s
166:	test: 0.8439941	best: 0.8439941 (166)	total: 2.73s	remaining: 21.8s
167:	test: 0.8440216	best: 0.8440216 (167)	total: 2.76s	remaining: 21.9s
168:	test: 0.8439941	best: 0.8440216 (167)	total: 2.77s	remaining: 21.8s
169:	test: 0.8439931	best: 0.8440216 (167)	total: 2.78s	remaining: 21.8s
170:	test: 0.8439998	best: 0.8440216 (167)	total: 2.79s	remaining: 21.7s
171:	test: 0.8440573	best: 0.8440573 (171)	total: 2.8s	remaining: 21.6s
172:	test: 0.8441173	best: 0.8441173 (172)	total: 2.84s	remaining: 21.8s
173:	test: 0.8442452	best: 0.8442452 (173)	total: 2.85s	remaining: 21.8s
174:	test: 0.8444657	best: 0.8444657 (174)	total: 2.9s	remaining: 22s
175:	test: 0.8447750	best: 0.8447750 (175)	total: 2.91s	remaining: 21.9s
176:	test: 0.8449961	best: 0.8449961 (176)	total: 2.92s	remaining: 21.9s
177:	test: 0.8451234	best: 0.8451234 (177)	total: 2.94s	remaining: 21.8s
178:	test: 0.8451436	best: 0.8451436 (178)	total: 2.95s	remaining: 21.8s
179:	test: 0.8452622	best: 0.8452622 (179)	total: 2.96s	remaining: 21.7s
180:	test: 0.8452875	best: 0.8452875 (180)	total: 2.97s	remaining: 21.7s
181:	test: 0.8453895	best: 0.8453895 (181)	total: 2.98s	remaining: 21.6s
182:	test: 0.8456680	best: 0.8456680 (182)	total: 3s	remaining: 21.6s
183:	test: 0.8457162	best: 0.8457162 (183)	total: 3.02s	remaining: 21.6s
184:	test: 0.8456556	best: 0.8457162 (183)	total: 3.03s	remaining: 21.6s
185:	test: 0.8456556	best: 0.8457162 (183)	total: 3.04s	remaining: 21.5s
186:	test: 0.8456867	best: 0.8457162 (183)	total: 3.06s	remaining: 21.5s
187:	test: 0.8457322	best: 0.8457322 (187)	total: 3.08s	remaining: 21.5s
188:	test: 0.8456755	best: 0.8457322 (187)	total: 3.1s	remaining: 21.5s
189:	test: 0.8456087	best: 0.8457322 (187)	total: 3.11s	remaining: 21.5s
190:	test: 0.8454301	best: 0.8457322 (187)	total: 3.13s	remaining: 21.4s
191:	test: 0.8454245	best: 0.8457322 (187)	total: 3.14s	remaining: 21.4s
192:	test: 0.8455244	best: 0.8457322 (187)	total: 3.15s	remaining: 21.3s
193:	test: 0.8455642	best: 0.8457322 (187)	total: 3.16s	remaining: 21.3s
194:	test: 0.8455534	best: 0.8457322 (187)	total: 3.17s	remaining: 21.2s
195:	test: 0.8456786	best: 0.8457322 (187)	total: 3.18s	remaining: 21.2s
196:	test: 0.8456553	best: 0.8457322 (187)	total: 3.19s	remaining: 21.1s
197:	test: 0.8455223	best: 0.8457322 (187)	total: 3.21s	remaining: 21.1s
198:	test: 0.8455083	best: 0.8457322 (187)	total: 3.22s	remaining: 21.1s
199:	test: 0.8454612	best: 0.8457322 (187)	total: 3.23s	remaining: 21s
200:	test: 0.8454540	best: 0.8457322 (187)	total: 3.24s	remaining: 20.9s
201:	test: 0.8454012	best: 0.8457322 (187)	total: 3.25s	remaining: 20.9s
202:	test: 0.8454721	best: 0.8457322 (187)	total: 3.29s	remaining: 21s
203:	test: 0.8454462	best: 0.8457322 (187)	total: 3.3s	remaining: 21s
204:	test: 0.8452836	best: 0.8457322 (187)	total: 3.32s	remaining: 20.9s
205:	test: 0.8452531	best: 0.8457322 (187)	total: 3.33s	remaining: 20.9s
206:	test: 0.8454058	best: 0.8457322 (187)	total: 3.34s	remaining: 20.8s
207:	test: 0.8453918	best: 0.8457322 (187)	total: 3.35s	remaining: 20.8s
208:	test: 0.8453235	best: 0.8457322 (187)	total: 3.36s	remaining: 20.7s
209:	test: 0.8452842	best: 0.8457322 (187)	total: 3.37s	remaining: 20.7s
210:	test: 0.8454162	best: 0.8457322 (187)	total: 3.37s	remaining: 20.6s
211:	test: 0.8453049	best: 0.8457322 (187)	total: 3.39s	remaining: 20.6s
212:	test: 0.8452743	best: 0.8457322 (187)	total: 3.4s	remaining: 20.6s
213:	test: 0.8453240	best: 0.8457322 (187)	total: 3.41s	remaining: 20.5s
214:	test: 0.8453106	best: 0.8457322 (187)	total: 3.42s	remaining: 20.5s
215:	test: 0.8453401	best: 0.8457322 (187)	total: 3.44s	remaining: 20.4s
216:	test: 0.8453453	best: 0.8457322 (187)	total: 3.45s	remaining: 20.4s
217:	test: 0.8451506	best: 0.8457322 (187)	total: 3.46s	remaining: 20.3s
218:	test: 0.8452733	best: 0.8457322 (187)	total: 3.47s	remaining: 20.3s
219:	test: 0.8452666	best: 0.8457322 (187)	total: 3.48s	remaining: 20.3s
220:	test: 0.8451641	best: 0.8457322 (187)	total: 3.5s	remaining: 20.2s
221:	test: 0.8452050	best: 0.8457322 (187)	total: 3.51s	remaining: 20.2s
222:	test: 0.8452914	best: 0.8457322 (187)	total: 3.52s	remaining: 20.1s
223:	test: 0.8454105	best: 0.8457322 (187)	total: 3.53s	remaining: 20.1s
224:	test: 0.8457848	best: 0.8457848 (224)	total: 3.54s	remaining: 20.1s
225:	test: 0.8457128	best: 0.8457848 (224)	total: 3.56s	remaining: 20s
226:	test: 0.8456512	best: 0.8457848 (224)	total: 3.57s	remaining: 20s
227:	test: 0.8456496	best: 0.8457848 (224)	total: 3.59s	remaining: 20s
228:	test: 0.8456947	best: 0.8457848 (224)	total: 3.6s	remaining: 20s
229:	test: 0.8458422	best: 0.8458422 (229)	total: 3.61s	remaining: 20s
230:	test: 0.8457982	best: 0.8458422 (229)	total: 3.63s	remaining: 20s
231:	test: 0.8459949	best: 0.8459949 (231)	total: 3.64s	remaining: 19.9s
232:	test: 0.8459545	best: 0.8459949 (231)	total: 3.65s	remaining: 19.9s
233:	test: 0.8460793	best: 0.8460793 (233)	total: 3.66s	remaining: 19.8s
234:	test: 0.8461544	best: 0.8461544 (234)	total: 3.67s	remaining: 19.8s
235:	test: 0.8461927	best: 0.8461927 (235)	total: 3.69s	remaining: 19.8s
236:	test: 0.8462558	best: 0.8462558 (236)	total: 3.7s	remaining: 19.7s
237:	test: 0.8463304	best: 0.8463304 (237)	total: 3.71s	remaining: 19.7s
238:	test: 0.8463956	best: 0.8463956 (238)	total: 3.73s	remaining: 19.7s
239:	test: 0.8463971	best: 0.8463971 (239)	total: 3.75s	remaining: 19.7s
240:	test: 0.8463257	best: 0.8463971 (239)	total: 3.77s	remaining: 19.7s
241:	test: 0.8463014	best: 0.8463971 (239)	total: 3.78s	remaining: 19.6s
242:	test: 0.8463480	best: 0.8463971 (239)	total: 3.79s	remaining: 19.6s
243:	test: 0.8462926	best: 0.8463971 (239)	total: 3.8s	remaining: 19.6s
244:	test: 0.8463762	best: 0.8463971 (239)	total: 3.82s	remaining: 19.6s
245:	test: 0.8463710	best: 0.8463971 (239)	total: 3.83s	remaining: 19.5s
246:	test: 0.8463482	best: 0.8463971 (239)	total: 3.84s	remaining: 19.5s
247:	test: 0.8462633	best: 0.8463971 (239)	total: 3.85s	remaining: 19.5s
248:	test: 0.8465429	best: 0.8465429 (248)	total: 3.87s	remaining: 19.4s
249:	test: 0.8465517	best: 0.8465517 (249)	total: 3.88s	remaining: 19.4s
250:	test: 0.8465776	best: 0.8465776 (250)	total: 3.89s	remaining: 19.4s
251:	test: 0.8466376	best: 0.8466376 (251)	total: 3.91s	remaining: 19.4s
252:	test: 0.8467784	best: 0.8467784 (252)	total: 3.92s	remaining: 19.3s
253:	test: 0.8467515	best: 0.8467784 (252)	total: 3.94s	remaining: 19.3s
254:	test: 0.8468095	best: 0.8468095 (254)	total: 3.96s	remaining: 19.3s
255:	test: 0.8468260	best: 0.8468260 (255)	total: 3.96s	remaining: 19.3s
256:	test: 0.8469705	best: 0.8469705 (256)	total: 3.98s	remaining: 19.2s
257:	test: 0.8469829	best: 0.8469829 (257)	total: 3.99s	remaining: 19.2s
258:	test: 0.8469715	best: 0.8469829 (257)	total: 4s	remaining: 19.2s
259:	test: 0.8468121	best: 0.8469829 (257)	total: 4.01s	remaining: 19.1s
260:	test: 0.8467556	best: 0.8469829 (257)	total: 4.02s	remaining: 19.1s
261:	test: 0.8467841	best: 0.8469829 (257)	total: 4.03s	remaining: 19s
262:	test: 0.8467494	best: 0.8469829 (257)	total: 4.04s	remaining: 19s
263:	test: 0.8468224	best: 0.8469829 (257)	total: 4.05s	remaining: 19s
264:	test: 0.8469850	best: 0.8469850 (264)	total: 4.06s	remaining: 18.9s
265:	test: 0.8469627	best: 0.8469850 (264)	total: 4.08s	remaining: 18.9s
266:	test: 0.8468038	best: 0.8469850 (264)	total: 4.09s	remaining: 18.9s
267:	test: 0.8469140	best: 0.8469850 (264)	total: 4.1s	remaining: 18.8s
268:	test: 0.8470781	best: 0.8470781 (268)	total: 4.11s	remaining: 18.8s
269:	test: 0.8471356	best: 0.8471356 (269)	total: 4.13s	remaining: 18.8s
270:	test: 0.8471118	best: 0.8471356 (269)	total: 4.15s	remaining: 18.8s
271:	test: 0.8471268	best: 0.8471356 (269)	total: 4.16s	remaining: 18.8s
272:	test: 0.8471931	best: 0.8471931 (272)	total: 4.17s	remaining: 18.8s
273:	test: 0.8470579	best: 0.8471931 (272)	total: 4.18s	remaining: 18.7s
274:	test: 0.8470429	best: 0.8471931 (272)	total: 4.19s	remaining: 18.7s
275:	test: 0.8470756	best: 0.8471931 (272)	total: 4.21s	remaining: 18.7s
276:	test: 0.8469912	best: 0.8471931 (272)	total: 4.24s	remaining: 18.7s
277:	test: 0.8469772	best: 0.8471931 (272)	total: 4.3s	remaining: 18.9s
278:	test: 0.8472117	best: 0.8472117 (278)	total: 4.32s	remaining: 18.9s
279:	test: 0.8472629	best: 0.8472629 (279)	total: 4.34s	remaining: 18.9s
280:	test: 0.8472987	best: 0.8472987 (280)	total: 4.35s	remaining: 18.9s
281:	test: 0.8473318	best: 0.8473318 (281)	total: 4.36s	remaining: 18.8s
282:	test: 0.8474317	best: 0.8474317 (282)	total: 4.37s	remaining: 18.8s
283:	test: 0.8473944	best: 0.8474317 (282)	total: 4.38s	remaining: 18.8s
284:	test: 0.8473349	best: 0.8474317 (282)	total: 4.4s	remaining: 18.8s
285:	test: 0.8473696	best: 0.8474317 (282)	total: 4.43s	remaining: 18.8s
286:	test: 0.8476900	best: 0.8476900 (286)	total: 4.45s	remaining: 18.8s
287:	test: 0.8477599	best: 0.8477599 (287)	total: 4.46s	remaining: 18.8s
288:	test: 0.8478728	best: 0.8478728 (288)	total: 4.48s	remaining: 18.8s
289:	test: 0.8478759	best: 0.8478759 (289)	total: 4.5s	remaining: 18.8s
290:	test: 0.8479344	best: 0.8479344 (290)	total: 4.53s	remaining: 18.8s
291:	test: 0.8479390	best: 0.8479390 (291)	total: 4.54s	remaining: 18.8s
292:	test: 0.8479199	best: 0.8479390 (291)	total: 4.56s	remaining: 18.8s
293:	test: 0.8480120	best: 0.8480120 (293)	total: 4.61s	remaining: 18.9s
294:	test: 0.8480094	best: 0.8480120 (293)	total: 4.62s	remaining: 18.9s
295:	test: 0.8480539	best: 0.8480539 (295)	total: 4.63s	remaining: 18.8s
296:	test: 0.8480638	best: 0.8480638 (296)	total: 4.64s	remaining: 18.8s
297:	test: 0.8480576	best: 0.8480638 (296)	total: 4.65s	remaining: 18.8s
298:	test: 0.8480472	best: 0.8480638 (296)	total: 4.66s	remaining: 18.7s
299:	test: 0.8481828	best: 0.8481828 (299)	total: 4.68s	remaining: 18.7s
300:	test: 0.8481932	best: 0.8481932 (300)	total: 4.69s	remaining: 18.7s
301:	test: 0.8482010	best: 0.8482010 (301)	total: 4.7s	remaining: 18.7s
302:	test: 0.8482010	best: 0.8482010 (301)	total: 4.71s	remaining: 18.6s
303:	test: 0.8482004	best: 0.8482010 (301)	total: 4.74s	remaining: 18.6s
304:	test: 0.8481114	best: 0.8482010 (301)	total: 4.76s	remaining: 18.6s
305:	test: 0.8481425	best: 0.8482010 (301)	total: 4.77s	remaining: 18.6s
306:	test: 0.8481394	best: 0.8482010 (301)	total: 4.78s	remaining: 18.6s
307:	test: 0.8481601	best: 0.8482010 (301)	total: 4.79s	remaining: 18.5s
308:	test: 0.8481683	best: 0.8482010 (301)	total: 4.83s	remaining: 18.6s
309:	test: 0.8481632	best: 0.8482010 (301)	total: 4.84s	remaining: 18.6s
310:	test: 0.8480304	best: 0.8482010 (301)	total: 4.87s	remaining: 18.6s
311:	test: 0.8480972	best: 0.8482010 (301)	total: 4.88s	remaining: 18.6s
312:	test: 0.8480262	best: 0.8482010 (301)	total: 4.91s	remaining: 18.6s
313:	test: 0.8480190	best: 0.8482010 (301)	total: 4.93s	remaining: 18.6s
314:	test: 0.8479729	best: 0.8482010 (301)	total: 4.95s	remaining: 18.6s
315:	test: 0.8479181	best: 0.8482010 (301)	total: 4.96s	remaining: 18.6s
316:	test: 0.8479672	best: 0.8482010 (301)	total: 4.97s	remaining: 18.6s
317:	test: 0.8480640	best: 0.8482010 (301)	total: 5s	remaining: 18.6s
318:	test: 0.8480806	best: 0.8482010 (301)	total: 5.01s	remaining: 18.5s
319:	test: 0.8481924	best: 0.8482010 (301)	total: 5.02s	remaining: 18.5s
320:	test: 0.8482209	best: 0.8482209 (320)	total: 5.03s	remaining: 18.5s
321:	test: 0.8479926	best: 0.8482209 (320)	total: 5.04s	remaining: 18.4s
322:	test: 0.8479843	best: 0.8482209 (320)	total: 5.06s	remaining: 18.4s
323:	test: 0.8480231	best: 0.8482209 (320)	total: 5.07s	remaining: 18.4s
324:	test: 0.8481841	best: 0.8482209 (320)	total: 5.08s	remaining: 18.4s
325:	test: 0.8481639	best: 0.8482209 (320)	total: 5.09s	remaining: 18.3s
326:	test: 0.8481557	best: 0.8482209 (320)	total: 5.1s	remaining: 18.3s
327:	test: 0.8481919	best: 0.8482209 (320)	total: 5.11s	remaining: 18.3s
328:	test: 0.8482002	best: 0.8482209 (320)	total: 5.14s	remaining: 18.3s
329:	test: 0.8482054	best: 0.8482209 (320)	total: 5.16s	remaining: 18.3s
330:	test: 0.8482338	best: 0.8482338 (330)	total: 5.17s	remaining: 18.3s
331:	test: 0.8481717	best: 0.8482338 (330)	total: 5.18s	remaining: 18.2s
332:	test: 0.8482002	best: 0.8482338 (330)	total: 5.21s	remaining: 18.3s
333:	test: 0.8482597	best: 0.8482597 (333)	total: 5.25s	remaining: 18.3s
334:	test: 0.8482100	best: 0.8482597 (333)	total: 5.26s	remaining: 18.3s
335:	test: 0.8482654	best: 0.8482654 (335)	total: 5.29s	remaining: 18.3s
336:	test: 0.8483348	best: 0.8483348 (336)	total: 5.31s	remaining: 18.3s
337:	test: 0.8482970	best: 0.8483348 (336)	total: 5.34s	remaining: 18.4s
338:	test: 0.8484471	best: 0.8484471 (338)	total: 5.35s	remaining: 18.3s
339:	test: 0.8483969	best: 0.8484471 (338)	total: 5.36s	remaining: 18.3s
340:	test: 0.8483674	best: 0.8484471 (338)	total: 5.38s	remaining: 18.3s
341:	test: 0.8483622	best: 0.8484471 (338)	total: 5.39s	remaining: 18.2s
342:	test: 0.8484409	best: 0.8484471 (338)	total: 5.41s	remaining: 18.2s
343:	test: 0.8484808	best: 0.8484808 (343)	total: 5.42s	remaining: 18.2s
344:	test: 0.8484616	best: 0.8484808 (343)	total: 5.44s	remaining: 18.2s
345:	test: 0.8484492	best: 0.8484808 (343)	total: 5.45s	remaining: 18.2s
346:	test: 0.8484083	best: 0.8484808 (343)	total: 5.47s	remaining: 18.2s
347:	test: 0.8483871	best: 0.8484808 (343)	total: 5.5s	remaining: 18.2s
348:	test: 0.8483829	best: 0.8484808 (343)	total: 5.51s	remaining: 18.2s
349:	test: 0.8483632	best: 0.8484808 (343)	total: 5.52s	remaining: 18.1s
350:	test: 0.8483638	best: 0.8484808 (343)	total: 5.53s	remaining: 18.1s
351:	test: 0.8484145	best: 0.8484808 (343)	total: 5.57s	remaining: 18.2s
352:	test: 0.8484911	best: 0.8484911 (352)	total: 5.59s	remaining: 18.2s
353:	test: 0.8485548	best: 0.8485548 (353)	total: 5.6s	remaining: 18.1s
354:	test: 0.8485879	best: 0.8485879 (354)	total: 5.61s	remaining: 18.1s
355:	test: 0.8486241	best: 0.8486241 (355)	total: 5.63s	remaining: 18.1s
356:	test: 0.8486790	best: 0.8486790 (356)	total: 5.64s	remaining: 18.1s
357:	test: 0.8487660	best: 0.8487660 (357)	total: 5.66s	remaining: 18s
358:	test: 0.8489637	best: 0.8489637 (358)	total: 5.69s	remaining: 18.1s
359:	test: 0.8488757	best: 0.8489637 (358)	total: 5.71s	remaining: 18.1s
360:	test: 0.8488550	best: 0.8489637 (358)	total: 5.72s	remaining: 18.1s
361:	test: 0.8487634	best: 0.8489637 (358)	total: 5.73s	remaining: 18s
362:	test: 0.8487582	best: 0.8489637 (358)	total: 5.74s	remaining: 18s
363:	test: 0.8487572	best: 0.8489637 (358)	total: 5.75s	remaining: 18s
364:	test: 0.8487515	best: 0.8489637 (358)	total: 5.77s	remaining: 17.9s
365:	test: 0.8487189	best: 0.8489637 (358)	total: 5.78s	remaining: 17.9s
366:	test: 0.8487562	best: 0.8489637 (358)	total: 5.8s	remaining: 17.9s
367:	test: 0.8488483	best: 0.8489637 (358)	total: 5.83s	remaining: 17.9s
368:	test: 0.8488100	best: 0.8489637 (358)	total: 5.85s	remaining: 17.9s
369:	test: 0.8488669	best: 0.8489637 (358)	total: 5.86s	remaining: 17.9s
370:	test: 0.8488980	best: 0.8489637 (358)	total: 5.88s	remaining: 17.9s
371:	test: 0.8488840	best: 0.8489637 (358)	total: 5.88s	remaining: 17.8s
372:	test: 0.8488416	best: 0.8489637 (358)	total: 5.9s	remaining: 17.8s
373:	test: 0.8488825	best: 0.8489637 (358)	total: 5.93s	remaining: 17.9s
374:	test: 0.8488121	best: 0.8489637 (358)	total: 5.95s	remaining: 17.8s
375:	test: 0.8488395	best: 0.8489637 (358)	total: 5.96s	remaining: 17.8s
376:	test: 0.8488167	best: 0.8489637 (358)	total: 5.98s	remaining: 17.8s
377:	test: 0.8488027	best: 0.8489637 (358)	total: 6s	remaining: 17.8s
378:	test: 0.8488747	best: 0.8489637 (358)	total: 6.03s	remaining: 17.8s
379:	test: 0.8487499	best: 0.8489637 (358)	total: 6.1s	remaining: 18s
380:	test: 0.8487375	best: 0.8489637 (358)	total: 6.11s	remaining: 18s
381:	test: 0.8487696	best: 0.8489637 (358)	total: 6.13s	remaining: 17.9s
382:	test: 0.8488752	best: 0.8489637 (358)	total: 6.16s	remaining: 18s
383:	test: 0.8488115	best: 0.8489637 (358)	total: 6.21s	remaining: 18s
384:	test: 0.8488685	best: 0.8489637 (358)	total: 6.22s	remaining: 18s
385:	test: 0.8488845	best: 0.8489637 (358)	total: 6.24s	remaining: 18s
386:	test: 0.8489415	best: 0.8489637 (358)	total: 6.3s	remaining: 18.1s
387:	test: 0.8489549	best: 0.8489637 (358)	total: 6.31s	remaining: 18.1s
388:	test: 0.8488788	best: 0.8489637 (358)	total: 6.33s	remaining: 18.1s
389:	test: 0.8488980	best: 0.8489637 (358)	total: 6.38s	remaining: 18.2s
390:	test: 0.8491071	best: 0.8491071 (390)	total: 6.42s	remaining: 18.2s
391:	test: 0.8490937	best: 0.8491071 (390)	total: 6.43s	remaining: 18.2s
392:	test: 0.8491299	best: 0.8491299 (392)	total: 6.44s	remaining: 18.1s
393:	test: 0.8492945	best: 0.8492945 (393)	total: 6.46s	remaining: 18.1s
394:	test: 0.8493458	best: 0.8493458 (394)	total: 6.47s	remaining: 18.1s
395:	test: 0.8493799	best: 0.8493799 (395)	total: 6.48s	remaining: 18.1s
396:	test: 0.8493598	best: 0.8493799 (395)	total: 6.5s	remaining: 18.1s
397:	test: 0.8494110	best: 0.8494110 (397)	total: 6.55s	remaining: 18.1s
398:	test: 0.8494566	best: 0.8494566 (398)	total: 6.56s	remaining: 18.1s
399:	test: 0.8494710	best: 0.8494710 (399)	total: 6.59s	remaining: 18.1s
400:	test: 0.8494581	best: 0.8494710 (399)	total: 6.61s	remaining: 18.1s
401:	test: 0.8493623	best: 0.8494710 (399)	total: 6.63s	remaining: 18.1s
402:	test: 0.8495176	best: 0.8495176 (402)	total: 6.66s	remaining: 18.1s
403:	test: 0.8495223	best: 0.8495223 (403)	total: 6.68s	remaining: 18.1s
404:	test: 0.8495570	best: 0.8495570 (404)	total: 6.74s	remaining: 18.2s
405:	test: 0.8495554	best: 0.8495570 (404)	total: 6.76s	remaining: 18.2s
406:	test: 0.8495518	best: 0.8495570 (404)	total: 6.78s	remaining: 18.2s
407:	test: 0.8495471	best: 0.8495570 (404)	total: 6.83s	remaining: 18.3s
408:	test: 0.8495228	best: 0.8495570 (404)	total: 6.84s	remaining: 18.3s
409:	test: 0.8494669	best: 0.8495570 (404)	total: 6.86s	remaining: 18.2s
410:	test: 0.8494664	best: 0.8495570 (404)	total: 6.89s	remaining: 18.3s
411:	test: 0.8494669	best: 0.8495570 (404)	total: 6.9s	remaining: 18.2s
412:	test: 0.8495482	best: 0.8495570 (404)	total: 6.92s	remaining: 18.2s
413:	test: 0.8495932	best: 0.8495932 (413)	total: 6.98s	remaining: 18.3s
414:	test: 0.8495083	best: 0.8495932 (413)	total: 7.01s	remaining: 18.3s
415:	test: 0.8495000	best: 0.8495932 (413)	total: 7.06s	remaining: 18.4s
416:	test: 0.8494985	best: 0.8495932 (413)	total: 7.07s	remaining: 18.4s
417:	test: 0.8495073	best: 0.8495932 (413)	total: 7.08s	remaining: 18.3s
418:	test: 0.8494923	best: 0.8495932 (413)	total: 7.09s	remaining: 18.3s
419:	test: 0.8494524	best: 0.8495932 (413)	total: 7.1s	remaining: 18.3s
420:	test: 0.8494830	best: 0.8495932 (413)	total: 7.11s	remaining: 18.2s
421:	test: 0.8494969	best: 0.8495932 (413)	total: 7.12s	remaining: 18.2s
422:	test: 0.8495197	best: 0.8495932 (413)	total: 7.13s	remaining: 18.2s
423:	test: 0.8495083	best: 0.8495932 (413)	total: 7.14s	remaining: 18.1s
424:	test: 0.8495182	best: 0.8495932 (413)	total: 7.15s	remaining: 18.1s
425:	test: 0.8495042	best: 0.8495932 (413)	total: 7.16s	remaining: 18.1s
426:	test: 0.8494804	best: 0.8495932 (413)	total: 7.18s	remaining: 18.1s
427:	test: 0.8494752	best: 0.8495932 (413)	total: 7.2s	remaining: 18s
428:	test: 0.8494804	best: 0.8495932 (413)	total: 7.22s	remaining: 18s
429:	test: 0.8495482	best: 0.8495932 (413)	total: 7.24s	remaining: 18s
430:	test: 0.8495497	best: 0.8495932 (413)	total: 7.26s	remaining: 18s
431:	test: 0.8495332	best: 0.8495932 (413)	total: 7.32s	remaining: 18.1s
432:	test: 0.8494990	best: 0.8495932 (413)	total: 7.36s	remaining: 18.1s
433:	test: 0.8494943	best: 0.8495932 (413)	total: 7.38s	remaining: 18.1s
434:	test: 0.8494855	best: 0.8495932 (413)	total: 7.4s	remaining: 18.1s
435:	test: 0.8494954	best: 0.8495932 (413)	total: 7.42s	remaining: 18.1s
436:	test: 0.8494990	best: 0.8495932 (413)	total: 7.45s	remaining: 18.1s
437:	test: 0.8494897	best: 0.8495932 (413)	total: 7.46s	remaining: 18.1s
438:	test: 0.8494622	best: 0.8495932 (413)	total: 7.47s	remaining: 18.1s
439:	test: 0.8494648	best: 0.8495932 (413)	total: 7.49s	remaining: 18s
440:	test: 0.8494591	best: 0.8495932 (413)	total: 7.5s	remaining: 18s
441:	test: 0.8494607	best: 0.8495932 (413)	total: 7.51s	remaining: 18s
442:	test: 0.8495197	best: 0.8495932 (413)	total: 7.59s	remaining: 18.1s
443:	test: 0.8495166	best: 0.8495932 (413)	total: 7.63s	remaining: 18.2s
444:	test: 0.8494886	best: 0.8495932 (413)	total: 7.67s	remaining: 18.2s
445:	test: 0.8494897	best: 0.8495932 (413)	total: 7.7s	remaining: 18.2s
446:	test: 0.8494742	best: 0.8495932 (413)	total: 7.71s	remaining: 18.2s
447:	test: 0.8494912	best: 0.8495932 (413)	total: 7.72s	remaining: 18.1s
448:	test: 0.8493541	best: 0.8495932 (413)	total: 7.74s	remaining: 18.1s
449:	test: 0.8493541	best: 0.8495932 (413)	total: 7.75s	remaining: 18.1s
450:	test: 0.8493515	best: 0.8495932 (413)	total: 7.75s	remaining: 18s
451:	test: 0.8493375	best: 0.8495932 (413)	total: 7.76s	remaining: 18s
452:	test: 0.8493194	best: 0.8495932 (413)	total: 7.78s	remaining: 18s
453:	test: 0.8493634	best: 0.8495932 (413)	total: 7.8s	remaining: 18s
454:	test: 0.8493763	best: 0.8495932 (413)	total: 7.81s	remaining: 17.9s
455:	test: 0.8493779	best: 0.8495932 (413)	total: 7.82s	remaining: 17.9s
456:	test: 0.8494182	best: 0.8495932 (413)	total: 7.83s	remaining: 17.9s
457:	test: 0.8495327	best: 0.8495932 (413)	total: 7.84s	remaining: 17.8s
458:	test: 0.8495047	best: 0.8495932 (413)	total: 7.85s	remaining: 17.8s
459:	test: 0.8495492	best: 0.8495932 (413)	total: 7.88s	remaining: 17.8s
460:	test: 0.8494162	best: 0.8495932 (413)	total: 7.89s	remaining: 17.8s
461:	test: 0.8494193	best: 0.8495932 (413)	total: 7.92s	remaining: 17.8s
462:	test: 0.8495047	best: 0.8495932 (413)	total: 7.94s	remaining: 17.8s
463:	test: 0.8495042	best: 0.8495932 (413)	total: 7.95s	remaining: 17.8s
464:	test: 0.8493665	best: 0.8495932 (413)	total: 7.97s	remaining: 17.8s
465:	test: 0.8493302	best: 0.8495932 (413)	total: 7.99s	remaining: 17.7s
466:	test: 0.8493111	best: 0.8495932 (413)	total: 8.01s	remaining: 17.7s
467:	test: 0.8493085	best: 0.8495932 (413)	total: 8.02s	remaining: 17.7s
468:	test: 0.8493028	best: 0.8495932 (413)	total: 8.04s	remaining: 17.7s
469:	test: 0.8492500	best: 0.8495932 (413)	total: 8.05s	remaining: 17.6s
470:	test: 0.8492495	best: 0.8495932 (413)	total: 8.06s	remaining: 17.6s
471:	test: 0.8491936	best: 0.8495932 (413)	total: 8.08s	remaining: 17.6s
472:	test: 0.8492303	best: 0.8495932 (413)	total: 8.1s	remaining: 17.6s
473:	test: 0.8491397	best: 0.8495932 (413)	total: 8.12s	remaining: 17.6s
474:	test: 0.8491910	best: 0.8495932 (413)	total: 8.19s	remaining: 17.7s
475:	test: 0.8492019	best: 0.8495932 (413)	total: 8.2s	remaining: 17.6s
476:	test: 0.8491962	best: 0.8495932 (413)	total: 8.21s	remaining: 17.6s
477:	test: 0.8492743	best: 0.8495932 (413)	total: 8.25s	remaining: 17.6s
478:	test: 0.8492899	best: 0.8495932 (413)	total: 8.27s	remaining: 17.6s
479:	test: 0.8493049	best: 0.8495932 (413)	total: 8.31s	remaining: 17.7s
480:	test: 0.8493271	best: 0.8495932 (413)	total: 8.32s	remaining: 17.6s
481:	test: 0.8493453	best: 0.8495932 (413)	total: 8.33s	remaining: 17.6s
482:	test: 0.8493561	best: 0.8495932 (413)	total: 8.34s	remaining: 17.6s
483:	test: 0.8493157	best: 0.8495932 (413)	total: 8.39s	remaining: 17.6s
484:	test: 0.8493178	best: 0.8495932 (413)	total: 8.4s	remaining: 17.6s
485:	test: 0.8493090	best: 0.8495932 (413)	total: 8.41s	remaining: 17.5s
486:	test: 0.8493261	best: 0.8495932 (413)	total: 8.42s	remaining: 17.5s
487:	test: 0.8493199	best: 0.8495932 (413)	total: 8.43s	remaining: 17.5s
488:	test: 0.8493132	best: 0.8495932 (413)	total: 8.45s	remaining: 17.5s
489:	test: 0.8493458	best: 0.8495932 (413)	total: 8.47s	remaining: 17.4s
490:	test: 0.8493354	best: 0.8495932 (413)	total: 8.48s	remaining: 17.4s
491:	test: 0.8492391	best: 0.8495932 (413)	total: 8.49s	remaining: 17.4s
492:	test: 0.8492345	best: 0.8495932 (413)	total: 8.5s	remaining: 17.4s
493:	test: 0.8492919	best: 0.8495932 (413)	total: 8.52s	remaining: 17.3s
494:	test: 0.8493437	best: 0.8495932 (413)	total: 8.53s	remaining: 17.3s
495:	test: 0.8493473	best: 0.8495932 (413)	total: 8.55s	remaining: 17.3s
496:	test: 0.8493696	best: 0.8495932 (413)	total: 8.56s	remaining: 17.3s
497:	test: 0.8493561	best: 0.8495932 (413)	total: 8.57s	remaining: 17.3s
498:	test: 0.8493577	best: 0.8495932 (413)	total: 8.59s	remaining: 17.2s
499:	test: 0.8493893	best: 0.8495932 (413)	total: 8.6s	remaining: 17.2s
500:	test: 0.8493913	best: 0.8495932 (413)	total: 8.61s	remaining: 17.2s
501:	test: 0.8494338	best: 0.8495932 (413)	total: 8.62s	remaining: 17.1s
502:	test: 0.8494317	best: 0.8495932 (413)	total: 8.63s	remaining: 17.1s
503:	test: 0.8494182	best: 0.8495932 (413)	total: 8.64s	remaining: 17.1s
504:	test: 0.8494048	best: 0.8495932 (413)	total: 8.65s	remaining: 17.1s
505:	test: 0.8494193	best: 0.8495932 (413)	total: 8.66s	remaining: 17s
506:	test: 0.8494001	best: 0.8495932 (413)	total: 8.68s	remaining: 17s
507:	test: 0.8493717	best: 0.8495932 (413)	total: 8.69s	remaining: 17s
508:	test: 0.8494074	best: 0.8495932 (413)	total: 8.7s	remaining: 16.9s
509:	test: 0.8493406	best: 0.8495932 (413)	total: 8.71s	remaining: 16.9s
510:	test: 0.8493354	best: 0.8495932 (413)	total: 8.72s	remaining: 16.9s
511:	test: 0.8493359	best: 0.8495932 (413)	total: 8.73s	remaining: 16.8s
512:	test: 0.8493696	best: 0.8495932 (413)	total: 8.74s	remaining: 16.8s
513:	test: 0.8494177	best: 0.8495932 (413)	total: 8.75s	remaining: 16.8s
514:	test: 0.8494193	best: 0.8495932 (413)	total: 8.76s	remaining: 16.8s
515:	test: 0.8494193	best: 0.8495932 (413)	total: 8.76s	remaining: 16.7s
516:	test: 0.8494338	best: 0.8495932 (413)	total: 8.77s	remaining: 16.7s
517:	test: 0.8494260	best: 0.8495932 (413)	total: 8.79s	remaining: 16.7s
518:	test: 0.8494721	best: 0.8495932 (413)	total: 8.8s	remaining: 16.6s
519:	test: 0.8495000	best: 0.8495932 (413)	total: 8.81s	remaining: 16.6s
520:	test: 0.8494519	best: 0.8495932 (413)	total: 8.84s	remaining: 16.6s
521:	test: 0.8495591	best: 0.8495932 (413)	total: 8.85s	remaining: 16.6s
522:	test: 0.8494141	best: 0.8495932 (413)	total: 8.88s	remaining: 16.6s
523:	test: 0.8494131	best: 0.8495932 (413)	total: 8.89s	remaining: 16.6s
524:	test: 0.8494079	best: 0.8495932 (413)	total: 8.9s	remaining: 16.5s
525:	test: 0.8494115	best: 0.8495932 (413)	total: 8.91s	remaining: 16.5s
526:	test: 0.8494622	best: 0.8495932 (413)	total: 8.92s	remaining: 16.5s
527:	test: 0.8494654	best: 0.8495932 (413)	total: 8.93s	remaining: 16.4s
528:	test: 0.8494364	best: 0.8495932 (413)	total: 8.94s	remaining: 16.4s
529:	test: 0.8494483	best: 0.8495932 (413)	total: 8.95s	remaining: 16.4s
530:	test: 0.8494270	best: 0.8495932 (413)	total: 8.96s	remaining: 16.4s
531:	test: 0.8494467	best: 0.8495932 (413)	total: 8.98s	remaining: 16.3s
532:	test: 0.8494478	best: 0.8495932 (413)	total: 8.99s	remaining: 16.3s
533:	test: 0.8494669	best: 0.8495932 (413)	total: 9s	remaining: 16.3s
534:	test: 0.8494426	best: 0.8495932 (413)	total: 9.01s	remaining: 16.2s
535:	test: 0.8494084	best: 0.8495932 (413)	total: 9.03s	remaining: 16.2s
536:	test: 0.8494151	best: 0.8495932 (413)	total: 9.04s	remaining: 16.2s
537:	test: 0.8494974	best: 0.8495932 (413)	total: 9.05s	remaining: 16.2s
538:	test: 0.8495119	best: 0.8495932 (413)	total: 9.06s	remaining: 16.1s
539:	test: 0.8495663	best: 0.8495932 (413)	total: 9.07s	remaining: 16.1s
540:	test: 0.8495528	best: 0.8495932 (413)	total: 9.08s	remaining: 16.1s
541:	test: 0.8495756	best: 0.8495932 (413)	total: 9.09s	remaining: 16.1s
542:	test: 0.8495663	best: 0.8495932 (413)	total: 9.1s	remaining: 16s
543:	test: 0.8495559	best: 0.8495932 (413)	total: 9.11s	remaining: 16s
544:	test: 0.8495528	best: 0.8495932 (413)	total: 9.13s	remaining: 16s
545:	test: 0.8494886	best: 0.8495932 (413)	total: 9.14s	remaining: 16s
546:	test: 0.8495016	best: 0.8495932 (413)	total: 9.15s	remaining: 15.9s
547:	test: 0.8494716	best: 0.8495932 (413)	total: 9.16s	remaining: 15.9s
548:	test: 0.8494669	best: 0.8495932 (413)	total: 9.17s	remaining: 15.9s
549:	test: 0.8494555	best: 0.8495932 (413)	total: 9.19s	remaining: 15.9s
550:	test: 0.8494503	best: 0.8495932 (413)	total: 9.21s	remaining: 15.9s
551:	test: 0.8494182	best: 0.8495932 (413)	total: 9.22s	remaining: 15.8s
552:	test: 0.8494369	best: 0.8495932 (413)	total: 9.23s	remaining: 15.8s
553:	test: 0.8494415	best: 0.8495932 (413)	total: 9.24s	remaining: 15.8s
554:	test: 0.8494296	best: 0.8495932 (413)	total: 9.25s	remaining: 15.7s
555:	test: 0.8494457	best: 0.8495932 (413)	total: 9.26s	remaining: 15.7s
556:	test: 0.8494188	best: 0.8495932 (413)	total: 9.27s	remaining: 15.7s
557:	test: 0.8494053	best: 0.8495932 (413)	total: 9.28s	remaining: 15.7s
558:	test: 0.8494679	best: 0.8495932 (413)	total: 9.29s	remaining: 15.6s
559:	test: 0.8494721	best: 0.8495932 (413)	total: 9.3s	remaining: 15.6s
560:	test: 0.8494602	best: 0.8495932 (413)	total: 9.31s	remaining: 15.6s
561:	test: 0.8494602	best: 0.8495932 (413)	total: 9.32s	remaining: 15.6s
562:	test: 0.8494845	best: 0.8495932 (413)	total: 9.33s	remaining: 15.5s
563:	test: 0.8494918	best: 0.8495932 (413)	total: 9.34s	remaining: 15.5s
564:	test: 0.8494788	best: 0.8495932 (413)	total: 9.35s	remaining: 15.5s
565:	test: 0.8494767	best: 0.8495932 (413)	total: 9.36s	remaining: 15.4s
566:	test: 0.8494483	best: 0.8495932 (413)	total: 9.38s	remaining: 15.4s
567:	test: 0.8494358	best: 0.8495932 (413)	total: 9.39s	remaining: 15.4s
568:	test: 0.8494172	best: 0.8495932 (413)	total: 9.41s	remaining: 15.4s
569:	test: 0.8493955	best: 0.8495932 (413)	total: 9.44s	remaining: 15.4s
570:	test: 0.8493975	best: 0.8495932 (413)	total: 9.49s	remaining: 15.4s
571:	test: 0.8493639	best: 0.8495932 (413)	total: 9.5s	remaining: 15.4s
572:	test: 0.8493618	best: 0.8495932 (413)	total: 9.55s	remaining: 15.5s
573:	test: 0.8493566	best: 0.8495932 (413)	total: 9.57s	remaining: 15.4s
574:	test: 0.8493473	best: 0.8495932 (413)	total: 9.62s	remaining: 15.5s
575:	test: 0.8493437	best: 0.8495932 (413)	total: 9.69s	remaining: 15.5s
576:	test: 0.8493396	best: 0.8495932 (413)	total: 9.72s	remaining: 15.5s
577:	test: 0.8491988	best: 0.8495932 (413)	total: 9.74s	remaining: 15.5s
578:	test: 0.8492122	best: 0.8495932 (413)	total: 9.8s	remaining: 15.6s
579:	test: 0.8492122	best: 0.8495932 (413)	total: 9.83s	remaining: 15.6s
580:	test: 0.8492309	best: 0.8495932 (413)	total: 9.87s	remaining: 15.6s
581:	test: 0.8491853	best: 0.8495932 (413)	total: 9.95s	remaining: 15.7s
582:	test: 0.8491781	best: 0.8495932 (413)	total: 10s	remaining: 15.8s
583:	test: 0.8491480	best: 0.8495932 (413)	total: 10.1s	remaining: 15.8s
584:	test: 0.8492153	best: 0.8495932 (413)	total: 10.1s	remaining: 15.8s
585:	test: 0.8492246	best: 0.8495932 (413)	total: 10.1s	remaining: 15.8s
586:	test: 0.8492448	best: 0.8495932 (413)	total: 10.1s	remaining: 15.8s
587:	test: 0.8492355	best: 0.8495932 (413)	total: 10.2s	remaining: 15.7s
588:	test: 0.8492365	best: 0.8495932 (413)	total: 10.2s	remaining: 15.8s
589:	test: 0.8492365	best: 0.8495932 (413)	total: 10.3s	remaining: 15.9s
590:	test: 0.8492448	best: 0.8495932 (413)	total: 10.3s	remaining: 15.8s
591:	test: 0.8492474	best: 0.8495932 (413)	total: 10.4s	remaining: 15.9s
592:	test: 0.8492319	best: 0.8495932 (413)	total: 10.4s	remaining: 15.9s
593:	test: 0.8492158	best: 0.8495932 (413)	total: 10.4s	remaining: 15.9s
594:	test: 0.8491993	best: 0.8495932 (413)	total: 10.4s	remaining: 15.9s
595:	test: 0.8491729	best: 0.8495932 (413)	total: 10.5s	remaining: 15.9s
596:	test: 0.8491625	best: 0.8495932 (413)	total: 10.5s	remaining: 15.9s
597:	test: 0.8491931	best: 0.8495932 (413)	total: 10.6s	remaining: 15.9s
598:	test: 0.8491977	best: 0.8495932 (413)	total: 10.6s	remaining: 15.9s
599:	test: 0.8492583	best: 0.8495932 (413)	total: 10.6s	remaining: 15.9s
600:	test: 0.8492531	best: 0.8495932 (413)	total: 10.7s	remaining: 16s
601:	test: 0.8493064	best: 0.8495932 (413)	total: 10.8s	remaining: 16s
602:	test: 0.8493101	best: 0.8495932 (413)	total: 10.8s	remaining: 16s
603:	test: 0.8493049	best: 0.8495932 (413)	total: 10.8s	remaining: 16s
604:	test: 0.8493018	best: 0.8495932 (413)	total: 10.8s	remaining: 16s
605:	test: 0.8493064	best: 0.8495932 (413)	total: 10.9s	remaining: 16s
606:	test: 0.8492536	best: 0.8495932 (413)	total: 10.9s	remaining: 16s
607:	test: 0.8491496	best: 0.8495932 (413)	total: 10.9s	remaining: 16s
608:	test: 0.8491330	best: 0.8495932 (413)	total: 11s	remaining: 16.1s
609:	test: 0.8491180	best: 0.8495932 (413)	total: 11s	remaining: 16s
610:	test: 0.8490968	best: 0.8495932 (413)	total: 11s	remaining: 16s
611:	test: 0.8490973	best: 0.8495932 (413)	total: 11s	remaining: 16s
612:	test: 0.8490875	best: 0.8495932 (413)	total: 11.1s	remaining: 16s
613:	test: 0.8490900	best: 0.8495932 (413)	total: 11.1s	remaining: 16s
614:	test: 0.8491061	best: 0.8495932 (413)	total: 11.1s	remaining: 16s
615:	test: 0.8490921	best: 0.8495932 (413)	total: 11.2s	remaining: 16s
616:	test: 0.8491020	best: 0.8495932 (413)	total: 11.2s	remaining: 16s
617:	test: 0.8490963	best: 0.8495932 (413)	total: 11.2s	remaining: 16s
618:	test: 0.8491030	best: 0.8495932 (413)	total: 11.2s	remaining: 15.9s
619:	test: 0.8491491	best: 0.8495932 (413)	total: 11.3s	remaining: 16s
620:	test: 0.8490844	best: 0.8495932 (413)	total: 11.3s	remaining: 16s
621:	test: 0.8490709	best: 0.8495932 (413)	total: 11.3s	remaining: 15.9s
622:	test: 0.8491025	best: 0.8495932 (413)	total: 11.3s	remaining: 15.9s
623:	test: 0.8491139	best: 0.8495932 (413)	total: 11.3s	remaining: 15.9s
624:	test: 0.8490693	best: 0.8495932 (413)	total: 11.3s	remaining: 15.9s
625:	test: 0.8490471	best: 0.8495932 (413)	total: 11.4s	remaining: 15.9s
626:	test: 0.8490378	best: 0.8495932 (413)	total: 11.4s	remaining: 15.9s
627:	test: 0.8490372	best: 0.8495932 (413)	total: 11.5s	remaining: 15.9s
628:	test: 0.8490228	best: 0.8495932 (413)	total: 11.5s	remaining: 15.9s
629:	test: 0.8490321	best: 0.8495932 (413)	total: 11.5s	remaining: 15.9s
630:	test: 0.8490269	best: 0.8495932 (413)	total: 11.5s	remaining: 15.9s
631:	test: 0.8490460	best: 0.8495932 (413)	total: 11.6s	remaining: 15.9s
632:	test: 0.8490492	best: 0.8495932 (413)	total: 11.6s	remaining: 15.9s
633:	test: 0.8490316	best: 0.8495932 (413)	total: 11.6s	remaining: 15.9s
634:	test: 0.8490181	best: 0.8495932 (413)	total: 11.6s	remaining: 15.8s
635:	test: 0.8490222	best: 0.8495932 (413)	total: 11.7s	remaining: 15.8s
636:	test: 0.8490222	best: 0.8495932 (413)	total: 11.7s	remaining: 15.8s
637:	test: 0.8490295	best: 0.8495932 (413)	total: 11.7s	remaining: 15.8s
638:	test: 0.8490026	best: 0.8495932 (413)	total: 11.7s	remaining: 15.8s
639:	test: 0.8490062	best: 0.8495932 (413)	total: 11.7s	remaining: 15.8s
640:	test: 0.8490222	best: 0.8495932 (413)	total: 11.7s	remaining: 15.7s
641:	test: 0.8490393	best: 0.8495932 (413)	total: 11.8s	remaining: 15.7s
642:	test: 0.8490274	best: 0.8495932 (413)	total: 11.8s	remaining: 15.7s
643:	test: 0.8490538	best: 0.8495932 (413)	total: 11.8s	remaining: 15.7s
644:	test: 0.8490838	best: 0.8495932 (413)	total: 11.8s	remaining: 15.7s
645:	test: 0.8490833	best: 0.8495932 (413)	total: 11.8s	remaining: 15.6s
646:	test: 0.8490781	best: 0.8495932 (413)	total: 11.8s	remaining: 15.6s
647:	test: 0.8490626	best: 0.8495932 (413)	total: 11.9s	remaining: 15.6s
648:	test: 0.8490310	best: 0.8495932 (413)	total: 11.9s	remaining: 15.6s
649:	test: 0.8489984	best: 0.8495932 (413)	total: 11.9s	remaining: 15.6s
650:	test: 0.8490455	best: 0.8495932 (413)	total: 11.9s	remaining: 15.5s
651:	test: 0.8490735	best: 0.8495932 (413)	total: 11.9s	remaining: 15.5s
652:	test: 0.8490471	best: 0.8495932 (413)	total: 11.9s	remaining: 15.5s
653:	test: 0.8490642	best: 0.8495932 (413)	total: 12s	remaining: 15.5s
654:	test: 0.8490921	best: 0.8495932 (413)	total: 12.1s	remaining: 15.6s
655:	test: 0.8491227	best: 0.8495932 (413)	total: 12.1s	remaining: 15.6s
656:	test: 0.8491061	best: 0.8495932 (413)	total: 12.2s	remaining: 15.6s
657:	test: 0.8491434	best: 0.8495932 (413)	total: 12.2s	remaining: 15.6s
658:	test: 0.8491247	best: 0.8495932 (413)	total: 12.2s	remaining: 15.6s
659:	test: 0.8491278	best: 0.8495932 (413)	total: 12.2s	remaining: 15.5s
660:	test: 0.8491527	best: 0.8495932 (413)	total: 12.3s	remaining: 15.6s
661:	test: 0.8491537	best: 0.8495932 (413)	total: 12.3s	remaining: 15.6s
662:	test: 0.8491786	best: 0.8495932 (413)	total: 12.3s	remaining: 15.6s
663:	test: 0.8491397	best: 0.8495932 (413)	total: 12.4s	remaining: 15.6s
664:	test: 0.8491470	best: 0.8495932 (413)	total: 12.4s	remaining: 15.6s
665:	test: 0.8491760	best: 0.8495932 (413)	total: 12.4s	remaining: 15.6s
666:	test: 0.8491806	best: 0.8495932 (413)	total: 12.5s	remaining: 15.6s
667:	test: 0.8491703	best: 0.8495932 (413)	total: 12.5s	remaining: 15.5s
668:	test: 0.8492117	best: 0.8495932 (413)	total: 12.5s	remaining: 15.5s
669:	test: 0.8491946	best: 0.8495932 (413)	total: 12.5s	remaining: 15.5s
670:	test: 0.8492060	best: 0.8495932 (413)	total: 12.5s	remaining: 15.5s
671:	test: 0.8492019	best: 0.8495932 (413)	total: 12.6s	remaining: 15.5s
672:	test: 0.8492029	best: 0.8495932 (413)	total: 12.6s	remaining: 15.4s
673:	test: 0.8492019	best: 0.8495932 (413)	total: 12.6s	remaining: 15.5s
674:	test: 0.8492112	best: 0.8495932 (413)	total: 12.7s	remaining: 15.5s
675:	test: 0.8492112	best: 0.8495932 (413)	total: 12.7s	remaining: 15.4s
676:	test: 0.8492148	best: 0.8495932 (413)	total: 12.7s	remaining: 15.4s
677:	test: 0.8492873	best: 0.8495932 (413)	total: 12.7s	remaining: 15.4s
678:	test: 0.8492826	best: 0.8495932 (413)	total: 12.7s	remaining: 15.4s
679:	test: 0.8492821	best: 0.8495932 (413)	total: 12.8s	remaining: 15.4s
680:	test: 0.8492785	best: 0.8495932 (413)	total: 12.8s	remaining: 15.4s
681:	test: 0.8492692	best: 0.8495932 (413)	total: 12.8s	remaining: 15.3s
682:	test: 0.8492666	best: 0.8495932 (413)	total: 12.8s	remaining: 15.3s
683:	test: 0.8492764	best: 0.8495932 (413)	total: 12.9s	remaining: 15.4s
684:	test: 0.8493670	best: 0.8495932 (413)	total: 12.9s	remaining: 15.3s
685:	test: 0.8493468	best: 0.8495932 (413)	total: 12.9s	remaining: 15.4s
686:	test: 0.8493333	best: 0.8495932 (413)	total: 13s	remaining: 15.3s
687:	test: 0.8493271	best: 0.8495932 (413)	total: 13s	remaining: 15.3s
688:	test: 0.8492961	best: 0.8495932 (413)	total: 13s	remaining: 15.3s
689:	test: 0.8492831	best: 0.8495932 (413)	total: 13s	remaining: 15.3s
690:	test: 0.8492899	best: 0.8495932 (413)	total: 13s	remaining: 15.3s
691:	test: 0.8493380	best: 0.8495932 (413)	total: 13.1s	remaining: 15.3s
692:	test: 0.8493396	best: 0.8495932 (413)	total: 13.1s	remaining: 15.2s
693:	test: 0.8493344	best: 0.8495932 (413)	total: 13.1s	remaining: 15.2s
694:	test: 0.8493432	best: 0.8495932 (413)	total: 13.1s	remaining: 15.2s
695:	test: 0.8493313	best: 0.8495932 (413)	total: 13.2s	remaining: 15.2s
696:	test: 0.8493204	best: 0.8495932 (413)	total: 13.2s	remaining: 15.2s
697:	test: 0.8493422	best: 0.8495932 (413)	total: 13.2s	remaining: 15.2s
698:	test: 0.8493411	best: 0.8495932 (413)	total: 13.2s	remaining: 15.1s
699:	test: 0.8493701	best: 0.8495932 (413)	total: 13.2s	remaining: 15.1s
700:	test: 0.8493753	best: 0.8495932 (413)	total: 13.2s	remaining: 15.1s
701:	test: 0.8493468	best: 0.8495932 (413)	total: 13.3s	remaining: 15.1s
702:	test: 0.8493546	best: 0.8495932 (413)	total: 13.3s	remaining: 15.1s
703:	test: 0.8493753	best: 0.8495932 (413)	total: 13.4s	remaining: 15.1s
704:	test: 0.8493530	best: 0.8495932 (413)	total: 13.4s	remaining: 15.1s
705:	test: 0.8493598	best: 0.8495932 (413)	total: 13.4s	remaining: 15.1s
706:	test: 0.8493463	best: 0.8495932 (413)	total: 13.4s	remaining: 15s
707:	test: 0.8493178	best: 0.8495932 (413)	total: 13.5s	remaining: 15.1s
708:	test: 0.8493261	best: 0.8495932 (413)	total: 13.6s	remaining: 15.1s
709:	test: 0.8493049	best: 0.8495932 (413)	total: 13.6s	remaining: 15.1s
710:	test: 0.8493411	best: 0.8495932 (413)	total: 13.6s	remaining: 15.1s
711:	test: 0.8493106	best: 0.8495932 (413)	total: 13.6s	remaining: 15.1s
712:	test: 0.8492987	best: 0.8495932 (413)	total: 13.7s	remaining: 15.1s
713:	test: 0.8493261	best: 0.8495932 (413)	total: 13.7s	remaining: 15.1s
714:	test: 0.8493059	best: 0.8495932 (413)	total: 13.7s	remaining: 15.1s
715:	test: 0.8492997	best: 0.8495932 (413)	total: 13.7s	remaining: 15s
716:	test: 0.8492956	best: 0.8495932 (413)	total: 13.7s	remaining: 15s
717:	test: 0.8492878	best: 0.8495932 (413)	total: 13.8s	remaining: 15s
718:	test: 0.8492945	best: 0.8495932 (413)	total: 13.8s	remaining: 15s
719:	test: 0.8492987	best: 0.8495932 (413)	total: 13.8s	remaining: 14.9s
720:	test: 0.8492847	best: 0.8495932 (413)	total: 13.8s	remaining: 14.9s
721:	test: 0.8492862	best: 0.8495932 (413)	total: 13.8s	remaining: 14.9s
722:	test: 0.8492950	best: 0.8495932 (413)	total: 13.8s	remaining: 14.9s
723:	test: 0.8493297	best: 0.8495932 (413)	total: 13.9s	remaining: 14.9s
724:	test: 0.8493748	best: 0.8495932 (413)	total: 13.9s	remaining: 14.8s
725:	test: 0.8493742	best: 0.8495932 (413)	total: 13.9s	remaining: 14.8s
726:	test: 0.8493753	best: 0.8495932 (413)	total: 13.9s	remaining: 14.8s
727:	test: 0.8493898	best: 0.8495932 (413)	total: 14s	remaining: 14.8s
728:	test: 0.8493975	best: 0.8495932 (413)	total: 14s	remaining: 14.8s
729:	test: 0.8493918	best: 0.8495932 (413)	total: 14s	remaining: 14.8s
730:	test: 0.8493898	best: 0.8495932 (413)	total: 14.1s	remaining: 14.8s
731:	test: 0.8493836	best: 0.8495932 (413)	total: 14.1s	remaining: 14.8s
732:	test: 0.8493836	best: 0.8495932 (413)	total: 14.1s	remaining: 14.8s
733:	test: 0.8493779	best: 0.8495932 (413)	total: 14.1s	remaining: 14.7s
734:	test: 0.8494395	best: 0.8495932 (413)	total: 14.1s	remaining: 14.7s
735:	test: 0.8494529	best: 0.8495932 (413)	total: 14.2s	remaining: 14.7s
736:	test: 0.8494503	best: 0.8495932 (413)	total: 14.2s	remaining: 14.7s
737:	test: 0.8494560	best: 0.8495932 (413)	total: 14.2s	remaining: 14.7s
738:	test: 0.8494379	best: 0.8495932 (413)	total: 14.2s	remaining: 14.6s
739:	test: 0.8494224	best: 0.8495932 (413)	total: 14.2s	remaining: 14.6s
740:	test: 0.8494923	best: 0.8495932 (413)	total: 14.2s	remaining: 14.6s
741:	test: 0.8495026	best: 0.8495932 (413)	total: 14.2s	remaining: 14.6s
742:	test: 0.8495037	best: 0.8495932 (413)	total: 14.3s	remaining: 14.5s
743:	test: 0.8495104	best: 0.8495932 (413)	total: 14.3s	remaining: 14.5s
744:	test: 0.8495109	best: 0.8495932 (413)	total: 14.3s	remaining: 14.5s
745:	test: 0.8495839	best: 0.8495932 (413)	total: 14.3s	remaining: 14.5s
746:	test: 0.8495860	best: 0.8495932 (413)	total: 14.4s	remaining: 14.5s
747:	test: 0.8496036	best: 0.8496036 (747)	total: 14.4s	remaining: 14.5s
748:	test: 0.8496015	best: 0.8496036 (747)	total: 14.4s	remaining: 14.5s
749:	test: 0.8496031	best: 0.8496036 (747)	total: 14.4s	remaining: 14.4s
750:	test: 0.8496031	best: 0.8496036 (747)	total: 14.5s	remaining: 14.4s
751:	test: 0.8495999	best: 0.8496036 (747)	total: 14.5s	remaining: 14.4s
752:	test: 0.8495673	best: 0.8496036 (747)	total: 14.6s	remaining: 14.5s
753:	test: 0.8495844	best: 0.8496036 (747)	total: 14.6s	remaining: 14.4s
754:	test: 0.8496098	best: 0.8496098 (754)	total: 14.6s	remaining: 14.4s
755:	test: 0.8495994	best: 0.8496098 (754)	total: 14.7s	remaining: 14.4s
756:	test: 0.8496108	best: 0.8496108 (756)	total: 14.7s	remaining: 14.4s
757:	test: 0.8496150	best: 0.8496150 (757)	total: 14.7s	remaining: 14.4s
758:	test: 0.8496155	best: 0.8496155 (758)	total: 14.7s	remaining: 14.4s
759:	test: 0.8496186	best: 0.8496186 (759)	total: 14.8s	remaining: 14.4s
760:	test: 0.8496113	best: 0.8496186 (759)	total: 14.8s	remaining: 14.4s
761:	test: 0.8496020	best: 0.8496186 (759)	total: 14.8s	remaining: 14.3s
762:	test: 0.8496232	best: 0.8496232 (762)	total: 14.8s	remaining: 14.3s
763:	test: 0.8496134	best: 0.8496232 (762)	total: 14.9s	remaining: 14.4s
764:	test: 0.8496320	best: 0.8496320 (764)	total: 14.9s	remaining: 14.4s
765:	test: 0.8496351	best: 0.8496351 (765)	total: 15s	remaining: 14.3s
766:	test: 0.8496652	best: 0.8496652 (766)	total: 15s	remaining: 14.3s
767:	test: 0.8496062	best: 0.8496652 (766)	total: 15s	remaining: 14.3s
768:	test: 0.8496372	best: 0.8496652 (766)	total: 15s	remaining: 14.2s
769:	test: 0.8496005	best: 0.8496652 (766)	total: 15s	remaining: 14.2s
770:	test: 0.8495622	best: 0.8496652 (766)	total: 15s	remaining: 14.2s
771:	test: 0.8495477	best: 0.8496652 (766)	total: 15.1s	remaining: 14.2s
772:	test: 0.8495280	best: 0.8496652 (766)	total: 15.1s	remaining: 14.2s
773:	test: 0.8495440	best: 0.8496652 (766)	total: 15.1s	remaining: 14.1s
774:	test: 0.8496015	best: 0.8496652 (766)	total: 15.1s	remaining: 14.1s
775:	test: 0.8496134	best: 0.8496652 (766)	total: 15.1s	remaining: 14.1s
776:	test: 0.8496191	best: 0.8496652 (766)	total: 15.1s	remaining: 14.1s
777:	test: 0.8496108	best: 0.8496652 (766)	total: 15.1s	remaining: 14s
778:	test: 0.8495896	best: 0.8496652 (766)	total: 15.2s	remaining: 14s
779:	test: 0.8495855	best: 0.8496652 (766)	total: 15.2s	remaining: 14s
780:	test: 0.8495782	best: 0.8496652 (766)	total: 15.2s	remaining: 14s
781:	test: 0.8495782	best: 0.8496652 (766)	total: 15.2s	remaining: 14s
782:	test: 0.8495637	best: 0.8496652 (766)	total: 15.2s	remaining: 14s
783:	test: 0.8495767	best: 0.8496652 (766)	total: 15.3s	remaining: 13.9s
784:	test: 0.8495663	best: 0.8496652 (766)	total: 15.3s	remaining: 13.9s
785:	test: 0.8495746	best: 0.8496652 (766)	total: 15.4s	remaining: 14s
786:	test: 0.8495756	best: 0.8496652 (766)	total: 15.4s	remaining: 13.9s
787:	test: 0.8495896	best: 0.8496652 (766)	total: 15.4s	remaining: 13.9s
788:	test: 0.8495663	best: 0.8496652 (766)	total: 15.4s	remaining: 13.9s
789:	test: 0.8495456	best: 0.8496652 (766)	total: 15.4s	remaining: 13.8s
790:	test: 0.8495492	best: 0.8496652 (766)	total: 15.4s	remaining: 13.8s
791:	test: 0.8495352	best: 0.8496652 (766)	total: 15.4s	remaining: 13.8s
792:	test: 0.8495254	best: 0.8496652 (766)	total: 15.4s	remaining: 13.8s
793:	test: 0.8495383	best: 0.8496652 (766)	total: 15.4s	remaining: 13.7s
794:	test: 0.8495182	best: 0.8496652 (766)	total: 15.5s	remaining: 13.7s
795:	test: 0.8495202	best: 0.8496652 (766)	total: 15.5s	remaining: 13.7s
796:	test: 0.8495616	best: 0.8496652 (766)	total: 15.5s	remaining: 13.7s
797:	test: 0.8495337	best: 0.8496652 (766)	total: 15.5s	remaining: 13.6s
798:	test: 0.8496269	best: 0.8496652 (766)	total: 15.5s	remaining: 13.6s
799:	test: 0.8496362	best: 0.8496652 (766)	total: 15.5s	remaining: 13.6s
800:	test: 0.8496595	best: 0.8496652 (766)	total: 15.5s	remaining: 13.5s
801:	test: 0.8496672	best: 0.8496672 (801)	total: 15.5s	remaining: 13.5s
802:	test: 0.8496693	best: 0.8496693 (802)	total: 15.6s	remaining: 13.5s
803:	test: 0.8496553	best: 0.8496693 (802)	total: 15.6s	remaining: 13.5s
804:	test: 0.8496465	best: 0.8496693 (802)	total: 15.6s	remaining: 13.4s
805:	test: 0.8496460	best: 0.8496693 (802)	total: 15.6s	remaining: 13.4s
806:	test: 0.8496486	best: 0.8496693 (802)	total: 15.6s	remaining: 13.4s
807:	test: 0.8496522	best: 0.8496693 (802)	total: 15.6s	remaining: 13.4s
808:	test: 0.8496341	best: 0.8496693 (802)	total: 15.6s	remaining: 13.3s
809:	test: 0.8496253	best: 0.8496693 (802)	total: 15.6s	remaining: 13.3s
810:	test: 0.8495958	best: 0.8496693 (802)	total: 15.6s	remaining: 13.3s
811:	test: 0.8495979	best: 0.8496693 (802)	total: 15.7s	remaining: 13.3s
812:	test: 0.8495730	best: 0.8496693 (802)	total: 15.7s	remaining: 13.2s
813:	test: 0.8495834	best: 0.8496693 (802)	total: 15.7s	remaining: 13.2s
814:	test: 0.8495663	best: 0.8496693 (802)	total: 15.7s	remaining: 13.2s
815:	test: 0.8495306	best: 0.8496693 (802)	total: 15.7s	remaining: 13.2s
816:	test: 0.8495420	best: 0.8496693 (802)	total: 15.7s	remaining: 13.1s
817:	test: 0.8495420	best: 0.8496693 (802)	total: 15.7s	remaining: 13.1s
818:	test: 0.8495430	best: 0.8496693 (802)	total: 15.7s	remaining: 13.1s
819:	test: 0.8495616	best: 0.8496693 (802)	total: 15.8s	remaining: 13.1s
820:	test: 0.8495637	best: 0.8496693 (802)	total: 15.8s	remaining: 13s
821:	test: 0.8495606	best: 0.8496693 (802)	total: 15.8s	remaining: 13s
822:	test: 0.8495699	best: 0.8496693 (802)	total: 15.8s	remaining: 13s
823:	test: 0.8496258	best: 0.8496693 (802)	total: 15.8s	remaining: 13s
824:	test: 0.8496331	best: 0.8496693 (802)	total: 15.8s	remaining: 12.9s
825:	test: 0.8496704	best: 0.8496704 (825)	total: 15.8s	remaining: 12.9s
826:	test: 0.8496704	best: 0.8496704 (825)	total: 15.8s	remaining: 12.9s
827:	test: 0.8496512	best: 0.8496704 (825)	total: 15.9s	remaining: 12.9s
828:	test: 0.8496533	best: 0.8496704 (825)	total: 15.9s	remaining: 12.8s
829:	test: 0.8496455	best: 0.8496704 (825)	total: 15.9s	remaining: 12.8s
830:	test: 0.8496564	best: 0.8496704 (825)	total: 15.9s	remaining: 12.8s
831:	test: 0.8496760	best: 0.8496760 (831)	total: 15.9s	remaining: 12.8s
832:	test: 0.8496983	best: 0.8496983 (832)	total: 15.9s	remaining: 12.8s
833:	test: 0.8497273	best: 0.8497273 (833)	total: 15.9s	remaining: 12.7s
834:	test: 0.8497107	best: 0.8497273 (833)	total: 16s	remaining: 12.7s
835:	test: 0.8496833	best: 0.8497273 (833)	total: 16s	remaining: 12.7s
836:	test: 0.8496905	best: 0.8497273 (833)	total: 16s	remaining: 12.7s
837:	test: 0.8496931	best: 0.8497273 (833)	total: 16s	remaining: 12.6s
838:	test: 0.8496916	best: 0.8497273 (833)	total: 16s	remaining: 12.6s
839:	test: 0.8496776	best: 0.8497273 (833)	total: 16s	remaining: 12.6s
840:	test: 0.8496786	best: 0.8497273 (833)	total: 16s	remaining: 12.6s
841:	test: 0.8496724	best: 0.8497273 (833)	total: 16.1s	remaining: 12.5s
842:	test: 0.8496771	best: 0.8497273 (833)	total: 16.1s	remaining: 12.5s
843:	test: 0.8497050	best: 0.8497273 (833)	total: 16.1s	remaining: 12.5s
844:	test: 0.8497128	best: 0.8497273 (833)	total: 16.1s	remaining: 12.5s
845:	test: 0.8497123	best: 0.8497273 (833)	total: 16.1s	remaining: 12.5s
846:	test: 0.8496771	best: 0.8497273 (833)	total: 16.1s	remaining: 12.4s
847:	test: 0.8496610	best: 0.8497273 (833)	total: 16.1s	remaining: 12.4s
848:	test: 0.8496232	best: 0.8497273 (833)	total: 16.2s	remaining: 12.4s
849:	test: 0.8496590	best: 0.8497273 (833)	total: 16.2s	remaining: 12.4s
850:	test: 0.8496698	best: 0.8497273 (833)	total: 16.2s	remaining: 12.3s
851:	test: 0.8496683	best: 0.8497273 (833)	total: 16.2s	remaining: 12.3s
852:	test: 0.8496688	best: 0.8497273 (833)	total: 16.2s	remaining: 12.3s
853:	test: 0.8496600	best: 0.8497273 (833)	total: 16.2s	remaining: 12.3s
854:	test: 0.8496538	best: 0.8497273 (833)	total: 16.2s	remaining: 12.2s
855:	test: 0.8496357	best: 0.8497273 (833)	total: 16.2s	remaining: 12.2s
856:	test: 0.8496310	best: 0.8497273 (833)	total: 16.3s	remaining: 12.2s
857:	test: 0.8496155	best: 0.8497273 (833)	total: 16.3s	remaining: 12.2s
858:	test: 0.8496165	best: 0.8497273 (833)	total: 16.3s	remaining: 12.1s
859:	test: 0.8495834	best: 0.8497273 (833)	total: 16.3s	remaining: 12.1s
860:	test: 0.8495823	best: 0.8497273 (833)	total: 16.3s	remaining: 12.1s
861:	test: 0.8496227	best: 0.8497273 (833)	total: 16.3s	remaining: 12.1s
862:	test: 0.8496227	best: 0.8497273 (833)	total: 16.3s	remaining: 12.1s
863:	test: 0.8496305	best: 0.8497273 (833)	total: 16.3s	remaining: 12s
864:	test: 0.8496662	best: 0.8497273 (833)	total: 16.4s	remaining: 12s
865:	test: 0.8496905	best: 0.8497273 (833)	total: 16.4s	remaining: 12s
866:	test: 0.8496905	best: 0.8497273 (833)	total: 16.4s	remaining: 12s
867:	test: 0.8497133	best: 0.8497273 (833)	total: 16.4s	remaining: 11.9s
868:	test: 0.8496968	best: 0.8497273 (833)	total: 16.4s	remaining: 11.9s
869:	test: 0.8496978	best: 0.8497273 (833)	total: 16.4s	remaining: 11.9s
870:	test: 0.8497102	best: 0.8497273 (833)	total: 16.4s	remaining: 11.9s
871:	test: 0.8497351	best: 0.8497351 (871)	total: 16.4s	remaining: 11.8s
872:	test: 0.8497283	best: 0.8497351 (871)	total: 16.4s	remaining: 11.8s
873:	test: 0.8497278	best: 0.8497351 (871)	total: 16.5s	remaining: 11.8s
874:	test: 0.8496854	best: 0.8497351 (871)	total: 16.5s	remaining: 11.8s
875:	test: 0.8496828	best: 0.8497351 (871)	total: 16.5s	remaining: 11.7s
876:	test: 0.8496833	best: 0.8497351 (871)	total: 16.5s	remaining: 11.7s
877:	test: 0.8496962	best: 0.8497351 (871)	total: 16.5s	remaining: 11.7s
878:	test: 0.8496973	best: 0.8497351 (871)	total: 16.5s	remaining: 11.7s
879:	test: 0.8496698	best: 0.8497351 (871)	total: 16.5s	remaining: 11.6s
880:	test: 0.8496781	best: 0.8497351 (871)	total: 16.5s	remaining: 11.6s
881:	test: 0.8496719	best: 0.8497351 (871)	total: 16.6s	remaining: 11.6s
882:	test: 0.8496745	best: 0.8497351 (871)	total: 16.6s	remaining: 11.6s
883:	test: 0.8496574	best: 0.8497351 (871)	total: 16.6s	remaining: 11.6s
884:	test: 0.8496719	best: 0.8497351 (871)	total: 16.6s	remaining: 11.6s
885:	test: 0.8497040	best: 0.8497351 (871)	total: 16.7s	remaining: 11.6s
886:	test: 0.8497009	best: 0.8497351 (871)	total: 16.7s	remaining: 11.5s
887:	test: 0.8497024	best: 0.8497351 (871)	total: 16.7s	remaining: 11.5s
888:	test: 0.8497097	best: 0.8497351 (871)	total: 16.7s	remaining: 11.5s
889:	test: 0.8497175	best: 0.8497351 (871)	total: 16.8s	remaining: 11.5s
890:	test: 0.8497076	best: 0.8497351 (871)	total: 16.8s	remaining: 11.5s
891:	test: 0.8496926	best: 0.8497351 (871)	total: 16.8s	remaining: 11.4s
892:	test: 0.8497211	best: 0.8497351 (871)	total: 16.8s	remaining: 11.4s
893:	test: 0.8497035	best: 0.8497351 (871)	total: 16.9s	remaining: 11.4s
894:	test: 0.8496885	best: 0.8497351 (871)	total: 16.9s	remaining: 11.4s
895:	test: 0.8496874	best: 0.8497351 (871)	total: 16.9s	remaining: 11.4s
896:	test: 0.8496771	best: 0.8497351 (871)	total: 16.9s	remaining: 11.4s
897:	test: 0.8496714	best: 0.8497351 (871)	total: 17s	remaining: 11.4s
898:	test: 0.8496610	best: 0.8497351 (871)	total: 17s	remaining: 11.4s
899:	test: 0.8496460	best: 0.8497351 (871)	total: 17s	remaining: 11.3s
900:	test: 0.8496393	best: 0.8497351 (871)	total: 17s	remaining: 11.3s
901:	test: 0.8496383	best: 0.8497351 (871)	total: 17s	remaining: 11.3s
902:	test: 0.8496165	best: 0.8497351 (871)	total: 17.1s	remaining: 11.3s
903:	test: 0.8496823	best: 0.8497351 (871)	total: 17.1s	remaining: 11.3s
904:	test: 0.8496559	best: 0.8497351 (871)	total: 17.1s	remaining: 11.2s
905:	test: 0.8496445	best: 0.8497351 (871)	total: 17.1s	remaining: 11.2s
906:	test: 0.8496450	best: 0.8497351 (871)	total: 17.1s	remaining: 11.2s
907:	test: 0.8496621	best: 0.8497351 (871)	total: 17.2s	remaining: 11.2s
908:	test: 0.8496828	best: 0.8497351 (871)	total: 17.2s	remaining: 11.2s
909:	test: 0.8496864	best: 0.8497351 (871)	total: 17.2s	remaining: 11.1s
910:	test: 0.8497024	best: 0.8497351 (871)	total: 17.2s	remaining: 11.1s
911:	test: 0.8497154	best: 0.8497351 (871)	total: 17.2s	remaining: 11.1s
912:	test: 0.8496936	best: 0.8497351 (871)	total: 17.2s	remaining: 11.1s
913:	test: 0.8496993	best: 0.8497351 (871)	total: 17.3s	remaining: 11.1s
914:	test: 0.8497097	best: 0.8497351 (871)	total: 17.3s	remaining: 11s
915:	test: 0.8496859	best: 0.8497351 (871)	total: 17.3s	remaining: 11s
916:	test: 0.8496926	best: 0.8497351 (871)	total: 17.3s	remaining: 11s
917:	test: 0.8497092	best: 0.8497351 (871)	total: 17.3s	remaining: 11s
918:	test: 0.8497506	best: 0.8497506 (918)	total: 17.3s	remaining: 10.9s
919:	test: 0.8497537	best: 0.8497537 (919)	total: 17.3s	remaining: 10.9s
920:	test: 0.8497822	best: 0.8497822 (920)	total: 17.3s	remaining: 10.9s
921:	test: 0.8497770	best: 0.8497822 (920)	total: 17.3s	remaining: 10.9s
922:	test: 0.8498665	best: 0.8498665 (922)	total: 17.4s	remaining: 10.8s
923:	test: 0.8498552	best: 0.8498665 (922)	total: 17.4s	remaining: 10.8s
924:	test: 0.8498453	best: 0.8498665 (922)	total: 17.4s	remaining: 10.8s
925:	test: 0.8498526	best: 0.8498665 (922)	total: 17.4s	remaining: 10.8s
926:	test: 0.8498484	best: 0.8498665 (922)	total: 17.4s	remaining: 10.8s
927:	test: 0.8498655	best: 0.8498665 (922)	total: 17.4s	remaining: 10.7s
928:	test: 0.8498660	best: 0.8498665 (922)	total: 17.5s	remaining: 10.7s
929:	test: 0.8498572	best: 0.8498665 (922)	total: 17.5s	remaining: 10.7s
930:	test: 0.8498541	best: 0.8498665 (922)	total: 17.5s	remaining: 10.7s
931:	test: 0.8498583	best: 0.8498665 (922)	total: 17.6s	remaining: 10.7s
932:	test: 0.8498546	best: 0.8498665 (922)	total: 17.6s	remaining: 10.7s
933:	test: 0.8498546	best: 0.8498665 (922)	total: 17.6s	remaining: 10.7s
934:	test: 0.8498443	best: 0.8498665 (922)	total: 17.6s	remaining: 10.6s
935:	test: 0.8498489	best: 0.8498665 (922)	total: 17.6s	remaining: 10.6s
936:	test: 0.8498205	best: 0.8498665 (922)	total: 17.6s	remaining: 10.6s
937:	test: 0.8498215	best: 0.8498665 (922)	total: 17.6s	remaining: 10.6s
938:	test: 0.8498241	best: 0.8498665 (922)	total: 17.7s	remaining: 10.6s
939:	test: 0.8498308	best: 0.8498665 (922)	total: 17.7s	remaining: 10.5s
940:	test: 0.8498220	best: 0.8498665 (922)	total: 17.7s	remaining: 10.5s
941:	test: 0.8498117	best: 0.8498665 (922)	total: 17.7s	remaining: 10.5s
942:	test: 0.8498200	best: 0.8498665 (922)	total: 17.7s	remaining: 10.5s
943:	test: 0.8498272	best: 0.8498665 (922)	total: 17.7s	remaining: 10.4s
944:	test: 0.8498376	best: 0.8498665 (922)	total: 17.7s	remaining: 10.4s
945:	test: 0.8498577	best: 0.8498665 (922)	total: 17.7s	remaining: 10.4s
946:	test: 0.8498365	best: 0.8498665 (922)	total: 17.8s	remaining: 10.4s
947:	test: 0.8498236	best: 0.8498665 (922)	total: 17.8s	remaining: 10.4s
948:	test: 0.8497894	best: 0.8498665 (922)	total: 17.8s	remaining: 10.3s
949:	test: 0.8498246	best: 0.8498665 (922)	total: 17.8s	remaining: 10.3s
950:	test: 0.8498691	best: 0.8498691 (950)	total: 17.8s	remaining: 10.3s
951:	test: 0.8498702	best: 0.8498702 (951)	total: 17.9s	remaining: 10.3s
952:	test: 0.8498810	best: 0.8498810 (952)	total: 17.9s	remaining: 10.3s
953:	test: 0.8498810	best: 0.8498810 (952)	total: 17.9s	remaining: 10.2s
954:	test: 0.8498722	best: 0.8498810 (952)	total: 17.9s	remaining: 10.2s
955:	test: 0.8498904	best: 0.8498904 (955)	total: 17.9s	remaining: 10.2s
956:	test: 0.8498986	best: 0.8498986 (956)	total: 18s	remaining: 10.2s
957:	test: 0.8498945	best: 0.8498986 (956)	total: 18s	remaining: 10.2s
958:	test: 0.8498924	best: 0.8498986 (956)	total: 18s	remaining: 10.1s
959:	test: 0.8498935	best: 0.8498986 (956)	total: 18s	remaining: 10.1s
960:	test: 0.8499090	best: 0.8499090 (960)	total: 18s	remaining: 10.1s
961:	test: 0.8499105	best: 0.8499105 (961)	total: 18s	remaining: 10.1s
962:	test: 0.8499074	best: 0.8499105 (961)	total: 18s	remaining: 10.1s
963:	test: 0.8499074	best: 0.8499105 (961)	total: 18s	remaining: 10s
964:	test: 0.8498940	best: 0.8499105 (961)	total: 18s	remaining: 10s
965:	test: 0.8498821	best: 0.8499105 (961)	total: 18.1s	remaining: 9.98s
966:	test: 0.8498562	best: 0.8499105 (961)	total: 18.1s	remaining: 9.97s
967:	test: 0.8498562	best: 0.8499105 (961)	total: 18.1s	remaining: 9.95s
968:	test: 0.8498510	best: 0.8499105 (961)	total: 18.1s	remaining: 9.93s
969:	test: 0.8498562	best: 0.8499105 (961)	total: 18.1s	remaining: 9.91s
970:	test: 0.8498500	best: 0.8499105 (961)	total: 18.1s	remaining: 9.89s
971:	test: 0.8498598	best: 0.8499105 (961)	total: 18.2s	remaining: 9.87s
972:	test: 0.8498852	best: 0.8499105 (961)	total: 18.2s	remaining: 9.85s
973:	test: 0.8498826	best: 0.8499105 (961)	total: 18.2s	remaining: 9.82s
974:	test: 0.8498826	best: 0.8499105 (961)	total: 18.2s	remaining: 9.8s
975:	test: 0.8499385	best: 0.8499385 (975)	total: 18.2s	remaining: 9.78s
976:	test: 0.8499385	best: 0.8499385 (975)	total: 18.2s	remaining: 9.75s
977:	test: 0.8499416	best: 0.8499416 (977)	total: 18.2s	remaining: 9.73s
978:	test: 0.8499452	best: 0.8499452 (978)	total: 18.2s	remaining: 9.71s
979:	test: 0.8499416	best: 0.8499452 (978)	total: 18.3s	remaining: 9.69s
980:	test: 0.8499659	best: 0.8499659 (980)	total: 18.3s	remaining: 9.66s
981:	test: 0.8499685	best: 0.8499685 (981)	total: 18.3s	remaining: 9.64s
982:	test: 0.8499566	best: 0.8499685 (981)	total: 18.3s	remaining: 9.63s
983:	test: 0.8499587	best: 0.8499685 (981)	total: 18.3s	remaining: 9.61s
984:	test: 0.8499209	best: 0.8499685 (981)	total: 18.3s	remaining: 9.59s
985:	test: 0.8499162	best: 0.8499685 (981)	total: 18.3s	remaining: 9.56s
986:	test: 0.8499178	best: 0.8499685 (981)	total: 18.4s	remaining: 9.55s
987:	test: 0.8499235	best: 0.8499685 (981)	total: 18.4s	remaining: 9.53s
988:	test: 0.8499395	best: 0.8499685 (981)	total: 18.4s	remaining: 9.5s
989:	test: 0.8499369	best: 0.8499685 (981)	total: 18.4s	remaining: 9.5s
990:	test: 0.8499618	best: 0.8499685 (981)	total: 18.5s	remaining: 9.48s
991:	test: 0.8499727	best: 0.8499727 (991)	total: 18.5s	remaining: 9.46s
992:	test: 0.8500042	best: 0.8500042 (992)	total: 18.5s	remaining: 9.44s
993:	test: 0.8500053	best: 0.8500053 (993)	total: 18.5s	remaining: 9.42s
994:	test: 0.8499996	best: 0.8500053 (993)	total: 18.6s	remaining: 9.42s
995:	test: 0.8500001	best: 0.8500053 (993)	total: 18.6s	remaining: 9.39s
996:	test: 0.8500767	best: 0.8500767 (996)	total: 18.6s	remaining: 9.38s
997:	test: 0.8500788	best: 0.8500788 (997)	total: 18.6s	remaining: 9.35s
998:	test: 0.8500633	best: 0.8500788 (997)	total: 18.6s	remaining: 9.33s
999:	test: 0.8500534	best: 0.8500788 (997)	total: 18.6s	remaining: 9.31s
1000:	test: 0.8500348	best: 0.8500788 (997)	total: 18.7s	remaining: 9.3s
1001:	test: 0.8500229	best: 0.8500788 (997)	total: 18.7s	remaining: 9.28s
1002:	test: 0.8500151	best: 0.8500788 (997)	total: 18.7s	remaining: 9.27s
1003:	test: 0.8500265	best: 0.8500788 (997)	total: 18.8s	remaining: 9.27s
1004:	test: 0.8500229	best: 0.8500788 (997)	total: 18.8s	remaining: 9.25s
1005:	test: 0.8500032	best: 0.8500788 (997)	total: 18.8s	remaining: 9.24s
1006:	test: 0.8500084	best: 0.8500788 (997)	total: 18.9s	remaining: 9.23s
1007:	test: 0.8500079	best: 0.8500788 (997)	total: 18.9s	remaining: 9.22s
1008:	test: 0.8499991	best: 0.8500788 (997)	total: 18.9s	remaining: 9.2s
1009:	test: 0.8499887	best: 0.8500788 (997)	total: 18.9s	remaining: 9.18s
1010:	test: 0.8499758	best: 0.8500788 (997)	total: 18.9s	remaining: 9.16s
1011:	test: 0.8499737	best: 0.8500788 (997)	total: 18.9s	remaining: 9.14s
1012:	test: 0.8499281	best: 0.8500788 (997)	total: 19s	remaining: 9.12s
1013:	test: 0.8499297	best: 0.8500788 (997)	total: 19s	remaining: 9.1s
1014:	test: 0.8499447	best: 0.8500788 (997)	total: 19s	remaining: 9.07s
1015:	test: 0.8499545	best: 0.8500788 (997)	total: 19s	remaining: 9.05s
1016:	test: 0.8499551	best: 0.8500788 (997)	total: 19s	remaining: 9.03s
1017:	test: 0.8499566	best: 0.8500788 (997)	total: 19s	remaining: 9.01s
1018:	test: 0.8499577	best: 0.8500788 (997)	total: 19s	remaining: 8.99s
1019:	test: 0.8499633	best: 0.8500788 (997)	total: 19.1s	remaining: 8.97s
1020:	test: 0.8499592	best: 0.8500788 (997)	total: 19.1s	remaining: 8.95s
1021:	test: 0.8499540	best: 0.8500788 (997)	total: 19.1s	remaining: 8.95s
1022:	test: 0.8499633	best: 0.8500788 (997)	total: 19.1s	remaining: 8.92s
1023:	test: 0.8500369	best: 0.8500788 (997)	total: 19.2s	remaining: 8.91s
1024:	test: 0.8500358	best: 0.8500788 (997)	total: 19.2s	remaining: 8.89s
1025:	test: 0.8500099	best: 0.8500788 (997)	total: 19.2s	remaining: 8.87s
1026:	test: 0.8500053	best: 0.8500788 (997)	total: 19.2s	remaining: 8.85s
1027:	test: 0.8500162	best: 0.8500788 (997)	total: 19.2s	remaining: 8.83s
1028:	test: 0.8500058	best: 0.8500788 (997)	total: 19.2s	remaining: 8.81s
1029:	test: 0.8500322	best: 0.8500788 (997)	total: 19.3s	remaining: 8.79s
1030:	test: 0.8500710	best: 0.8500788 (997)	total: 19.3s	remaining: 8.77s
1031:	test: 0.8500809	best: 0.8500809 (1031)	total: 19.3s	remaining: 8.74s
1032:	test: 0.8500809	best: 0.8500809 (1031)	total: 19.3s	remaining: 8.72s
1033:	test: 0.8500664	best: 0.8500809 (1031)	total: 19.3s	remaining: 8.71s
1034:	test: 0.8500508	best: 0.8500809 (1031)	total: 19.3s	remaining: 8.68s
1035:	test: 0.8500519	best: 0.8500809 (1031)	total: 19.3s	remaining: 8.66s
1036:	test: 0.8501010	best: 0.8501010 (1036)	total: 19.3s	remaining: 8.64s
1037:	test: 0.8501109	best: 0.8501109 (1037)	total: 19.4s	remaining: 8.62s
1038:	test: 0.8501067	best: 0.8501109 (1037)	total: 19.4s	remaining: 8.59s
1039:	test: 0.8501171	best: 0.8501171 (1039)	total: 19.4s	remaining: 8.57s
1040:	test: 0.8501285	best: 0.8501285 (1040)	total: 19.4s	remaining: 8.55s
1041:	test: 0.8501730	best: 0.8501730 (1041)	total: 19.4s	remaining: 8.53s
1042:	test: 0.8501792	best: 0.8501792 (1042)	total: 19.4s	remaining: 8.51s
1043:	test: 0.8502191	best: 0.8502191 (1043)	total: 19.4s	remaining: 8.49s
1044:	test: 0.8501984	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.47s
1045:	test: 0.8501927	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.45s
1046:	test: 0.8501911	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.43s
1047:	test: 0.8501844	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.41s
1048:	test: 0.8501782	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.39s
1049:	test: 0.8501751	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.37s
1050:	test: 0.8501797	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.34s
1051:	test: 0.8501782	best: 0.8502191 (1043)	total: 19.5s	remaining: 8.32s
1052:	test: 0.8502600	best: 0.8502600 (1052)	total: 19.6s	remaining: 8.3s
1053:	test: 0.8502827	best: 0.8502827 (1053)	total: 19.6s	remaining: 8.28s
1054:	test: 0.8502708	best: 0.8502827 (1053)	total: 19.6s	remaining: 8.26s
1055:	test: 0.8502796	best: 0.8502827 (1053)	total: 19.6s	remaining: 8.24s
1056:	test: 0.8502890	best: 0.8502890 (1056)	total: 19.6s	remaining: 8.22s
1057:	test: 0.8503035	best: 0.8503035 (1057)	total: 19.6s	remaining: 8.2s
1058:	test: 0.8502910	best: 0.8503035 (1057)	total: 19.6s	remaining: 8.18s
1059:	test: 0.8502631	best: 0.8503035 (1057)	total: 19.6s	remaining: 8.15s
1060:	test: 0.8502817	best: 0.8503035 (1057)	total: 19.7s	remaining: 8.15s
1061:	test: 0.8502915	best: 0.8503035 (1057)	total: 19.7s	remaining: 8.13s
1062:	test: 0.8502900	best: 0.8503035 (1057)	total: 19.7s	remaining: 8.11s
1063:	test: 0.8502786	best: 0.8503035 (1057)	total: 19.7s	remaining: 8.08s
1064:	test: 0.8502760	best: 0.8503035 (1057)	total: 19.7s	remaining: 8.06s
1065:	test: 0.8503112	best: 0.8503112 (1065)	total: 19.7s	remaining: 8.04s
1066:	test: 0.8503029	best: 0.8503112 (1065)	total: 19.8s	remaining: 8.02s
1067:	test: 0.8503009	best: 0.8503112 (1065)	total: 19.8s	remaining: 8s
1068:	test: 0.8502900	best: 0.8503112 (1065)	total: 19.8s	remaining: 7.98s
1069:	test: 0.8502926	best: 0.8503112 (1065)	total: 19.8s	remaining: 7.96s
1070:	test: 0.8502895	best: 0.8503112 (1065)	total: 19.8s	remaining: 7.94s
1071:	test: 0.8502941	best: 0.8503112 (1065)	total: 19.8s	remaining: 7.92s
1072:	test: 0.8503092	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.9s
1073:	test: 0.8503081	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.88s
1074:	test: 0.8503081	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.86s
1075:	test: 0.8502941	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.84s
1076:	test: 0.8502926	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.83s
1077:	test: 0.8502884	best: 0.8503112 (1065)	total: 19.9s	remaining: 7.81s
1078:	test: 0.8502869	best: 0.8503112 (1065)	total: 20s	remaining: 7.79s
1079:	test: 0.8502843	best: 0.8503112 (1065)	total: 20s	remaining: 7.77s
1080:	test: 0.8502558	best: 0.8503112 (1065)	total: 20s	remaining: 7.75s
1081:	test: 0.8502481	best: 0.8503112 (1065)	total: 20s	remaining: 7.74s
1082:	test: 0.8502522	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.72s
1083:	test: 0.8502196	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.7s
1084:	test: 0.8502196	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.68s
1085:	test: 0.8502082	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.66s
1086:	test: 0.8502139	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.64s
1087:	test: 0.8502010	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.62s
1088:	test: 0.8502046	best: 0.8503112 (1065)	total: 20.1s	remaining: 7.59s
1089:	test: 0.8502004	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.58s
1090:	test: 0.8501984	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.56s
1091:	test: 0.8501839	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.54s
1092:	test: 0.8501896	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.52s
1093:	test: 0.8501502	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.5s
1094:	test: 0.8501378	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.47s
1095:	test: 0.8501896	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.46s
1096:	test: 0.8501839	best: 0.8503112 (1065)	total: 20.2s	remaining: 7.44s
1097:	test: 0.8501704	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.42s
1098:	test: 0.8501673	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.4s
1099:	test: 0.8501901	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.38s
1100:	test: 0.8501896	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.36s
1101:	test: 0.8501937	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.34s
1102:	test: 0.8501968	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.32s
1103:	test: 0.8502046	best: 0.8503112 (1065)	total: 20.3s	remaining: 7.3s
1104:	test: 0.8502450	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.28s
1105:	test: 0.8502470	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.25s
1106:	test: 0.8502201	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.24s
1107:	test: 0.8502346	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.22s
1108:	test: 0.8502926	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.2s
1109:	test: 0.8502796	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.18s
1110:	test: 0.8502724	best: 0.8503112 (1065)	total: 20.4s	remaining: 7.16s
1111:	test: 0.8502745	best: 0.8503112 (1065)	total: 20.5s	remaining: 7.14s
1112:	test: 0.8502765	best: 0.8503112 (1065)	total: 20.5s	remaining: 7.12s
1113:	test: 0.8502796	best: 0.8503112 (1065)	total: 20.5s	remaining: 7.11s
1114:	test: 0.8502750	best: 0.8503112 (1065)	total: 20.5s	remaining: 7.09s
1115:	test: 0.8502683	best: 0.8503112 (1065)	total: 20.5s	remaining: 7.07s
1116:	test: 0.8503123	best: 0.8503123 (1116)	total: 20.6s	remaining: 7.05s
1117:	test: 0.8502988	best: 0.8503123 (1116)	total: 20.6s	remaining: 7.03s
1118:	test: 0.8503014	best: 0.8503123 (1116)	total: 20.6s	remaining: 7.01s
1119:	test: 0.8502936	best: 0.8503123 (1116)	total: 20.6s	remaining: 6.99s
1120:	test: 0.8502890	best: 0.8503123 (1116)	total: 20.6s	remaining: 6.97s
1121:	test: 0.8502926	best: 0.8503123 (1116)	total: 20.6s	remaining: 6.95s
1122:	test: 0.8502895	best: 0.8503123 (1116)	total: 20.7s	remaining: 6.93s
1123:	test: 0.8502952	best: 0.8503123 (1116)	total: 20.7s	remaining: 6.91s
1124:	test: 0.8502941	best: 0.8503123 (1116)	total: 20.7s	remaining: 6.89s
1125:	test: 0.8503754	best: 0.8503754 (1125)	total: 20.7s	remaining: 6.87s
1126:	test: 0.8503790	best: 0.8503790 (1126)	total: 20.7s	remaining: 6.86s
1127:	test: 0.8504246	best: 0.8504246 (1127)	total: 20.7s	remaining: 6.84s
1128:	test: 0.8504199	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.82s
1129:	test: 0.8504241	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.8s
1130:	test: 0.8504236	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.78s
1131:	test: 0.8504220	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.76s
1132:	test: 0.8504148	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.74s
1133:	test: 0.8504199	best: 0.8504246 (1127)	total: 20.8s	remaining: 6.72s
1134:	test: 0.8504386	best: 0.8504386 (1134)	total: 20.8s	remaining: 6.69s
1135:	test: 0.8504380	best: 0.8504386 (1134)	total: 20.8s	remaining: 6.67s
1136:	test: 0.8504380	best: 0.8504386 (1134)	total: 20.8s	remaining: 6.65s
1137:	test: 0.8504349	best: 0.8504386 (1134)	total: 20.8s	remaining: 6.63s
1138:	test: 0.8504391	best: 0.8504391 (1138)	total: 20.9s	remaining: 6.61s
1139:	test: 0.8504427	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.59s
1140:	test: 0.8504365	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.57s
1141:	test: 0.8504179	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.55s
1142:	test: 0.8504375	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.53s
1143:	test: 0.8504075	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.51s
1144:	test: 0.8504034	best: 0.8504427 (1139)	total: 20.9s	remaining: 6.49s
1145:	test: 0.8504173	best: 0.8504427 (1139)	total: 21s	remaining: 6.47s
1146:	test: 0.8504080	best: 0.8504427 (1139)	total: 21s	remaining: 6.45s
1147:	test: 0.8504091	best: 0.8504427 (1139)	total: 21s	remaining: 6.43s
1148:	test: 0.8504039	best: 0.8504427 (1139)	total: 21s	remaining: 6.41s
1149:	test: 0.8503708	best: 0.8504427 (1139)	total: 21s	remaining: 6.39s
1150:	test: 0.8503500	best: 0.8504427 (1139)	total: 21s	remaining: 6.37s
1151:	test: 0.8503242	best: 0.8504427 (1139)	total: 21s	remaining: 6.35s
1152:	test: 0.8503019	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.33s
1153:	test: 0.8503071	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.31s
1154:	test: 0.8503066	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.29s
1155:	test: 0.8503060	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.27s
1156:	test: 0.8503086	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.25s
1157:	test: 0.8503097	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.24s
1158:	test: 0.8502957	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.22s
1159:	test: 0.8503304	best: 0.8504427 (1139)	total: 21.1s	remaining: 6.2s
1160:	test: 0.8503304	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.18s
1161:	test: 0.8503309	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.16s
1162:	test: 0.8503174	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.14s
1163:	test: 0.8503205	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.12s
1164:	test: 0.8503330	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.09s
1165:	test: 0.8503304	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.08s
1166:	test: 0.8503381	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.06s
1167:	test: 0.8503185	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.04s
1168:	test: 0.8503117	best: 0.8504427 (1139)	total: 21.2s	remaining: 6.01s
1169:	test: 0.8503148	best: 0.8504427 (1139)	total: 21.3s	remaining: 6s
1170:	test: 0.8503097	best: 0.8504427 (1139)	total: 21.3s	remaining: 5.98s
1171:	test: 0.8502947	best: 0.8504427 (1139)	total: 21.3s	remaining: 5.96s
1172:	test: 0.8503154	best: 0.8504427 (1139)	total: 21.3s	remaining: 5.94s
1173:	test: 0.8503159	best: 0.8504427 (1139)	total: 21.3s	remaining: 5.92s
1174:	test: 0.8503154	best: 0.8504427 (1139)	total: 21.3s	remaining: 5.9s
1175:	test: 0.8502838	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.88s
1176:	test: 0.8502952	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.86s
1177:	test: 0.8503827	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.84s
1178:	test: 0.8503920	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.82s
1179:	test: 0.8503863	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.8s
1180:	test: 0.8503739	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.78s
1181:	test: 0.8503733	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.76s
1182:	test: 0.8503682	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.74s
1183:	test: 0.8503661	best: 0.8504427 (1139)	total: 21.4s	remaining: 5.72s
1184:	test: 0.8503790	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.7s
1185:	test: 0.8503775	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.68s
1186:	test: 0.8503780	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.66s
1187:	test: 0.8503796	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.65s
1188:	test: 0.8503780	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.63s
1189:	test: 0.8503713	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.61s
1190:	test: 0.8503718	best: 0.8504427 (1139)	total: 21.5s	remaining: 5.59s
1191:	test: 0.8503733	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.57s
1192:	test: 0.8503749	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.55s
1193:	test: 0.8503837	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.53s
1194:	test: 0.8504060	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.51s
1195:	test: 0.8504132	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.49s
1196:	test: 0.8504220	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.47s
1197:	test: 0.8504153	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.45s
1198:	test: 0.8504085	best: 0.8504427 (1139)	total: 21.6s	remaining: 5.43s
1199:	test: 0.8504510	best: 0.8504510 (1199)	total: 21.7s	remaining: 5.41s
1200:	test: 0.8504515	best: 0.8504515 (1200)	total: 21.7s	remaining: 5.4s
1201:	test: 0.8504510	best: 0.8504515 (1200)	total: 21.7s	remaining: 5.38s
1202:	test: 0.8504593	best: 0.8504593 (1202)	total: 21.7s	remaining: 5.36s
1203:	test: 0.8504598	best: 0.8504598 (1203)	total: 21.7s	remaining: 5.34s
1204:	test: 0.8504551	best: 0.8504598 (1203)	total: 21.7s	remaining: 5.32s
1205:	test: 0.8504479	best: 0.8504598 (1203)	total: 21.7s	remaining: 5.3s
1206:	test: 0.8504551	best: 0.8504598 (1203)	total: 21.8s	remaining: 5.28s
1207:	test: 0.8504619	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.26s
1208:	test: 0.8504380	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.24s
1209:	test: 0.8504386	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.22s
1210:	test: 0.8504613	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.2s
1211:	test: 0.8504458	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.19s
1212:	test: 0.8504386	best: 0.8504619 (1207)	total: 21.8s	remaining: 5.17s
1213:	test: 0.8504515	best: 0.8504619 (1207)	total: 21.9s	remaining: 5.15s
1214:	test: 0.8504432	best: 0.8504619 (1207)	total: 21.9s	remaining: 5.13s
1215:	test: 0.8504401	best: 0.8504619 (1207)	total: 21.9s	remaining: 5.11s
1216:	test: 0.8504634	best: 0.8504634 (1216)	total: 21.9s	remaining: 5.09s
1217:	test: 0.8504577	best: 0.8504634 (1216)	total: 21.9s	remaining: 5.07s
1218:	test: 0.8504598	best: 0.8504634 (1216)	total: 21.9s	remaining: 5.05s
1219:	test: 0.8504676	best: 0.8504676 (1219)	total: 21.9s	remaining: 5.03s
1220:	test: 0.8504883	best: 0.8504883 (1220)	total: 21.9s	remaining: 5.01s
1221:	test: 0.8504774	best: 0.8504883 (1220)	total: 22s	remaining: 5s
1222:	test: 0.8504831	best: 0.8504883 (1220)	total: 22s	remaining: 4.98s
1223:	test: 0.8504691	best: 0.8504883 (1220)	total: 22s	remaining: 4.96s
1224:	test: 0.8504670	best: 0.8504883 (1220)	total: 22s	remaining: 4.94s
1225:	test: 0.8504784	best: 0.8504883 (1220)	total: 22s	remaining: 4.92s
1226:	test: 0.8504784	best: 0.8504883 (1220)	total: 22s	remaining: 4.9s
1227:	test: 0.8504841	best: 0.8504883 (1220)	total: 22s	remaining: 4.88s
1228:	test: 0.8504536	best: 0.8504883 (1220)	total: 22s	remaining: 4.86s
1229:	test: 0.8504789	best: 0.8504883 (1220)	total: 22.1s	remaining: 4.84s
1230:	test: 0.8504676	best: 0.8504883 (1220)	total: 22.1s	remaining: 4.82s
1231:	test: 0.8505607	best: 0.8505607 (1231)	total: 22.1s	remaining: 4.8s
1232:	test: 0.8505499	best: 0.8505607 (1231)	total: 22.1s	remaining: 4.79s
1233:	test: 0.8505250	best: 0.8505607 (1231)	total: 22.1s	remaining: 4.77s
1234:	test: 0.8505297	best: 0.8505607 (1231)	total: 22.1s	remaining: 4.75s
1235:	test: 0.8505271	best: 0.8505607 (1231)	total: 22.1s	remaining: 4.73s
1236:	test: 0.8505204	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.71s
1237:	test: 0.8505110	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.69s
1238:	test: 0.8505038	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.67s
1239:	test: 0.8504965	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.65s
1240:	test: 0.8505085	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.63s
1241:	test: 0.8505038	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.61s
1242:	test: 0.8505167	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.59s
1243:	test: 0.8505017	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.58s
1244:	test: 0.8505121	best: 0.8505607 (1231)	total: 22.2s	remaining: 4.56s
1245:	test: 0.8505121	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.54s
1246:	test: 0.8505178	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.52s
1247:	test: 0.8505141	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.5s
1248:	test: 0.8505229	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.48s
1249:	test: 0.8505110	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.46s
1250:	test: 0.8505017	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.44s
1251:	test: 0.8504991	best: 0.8505607 (1231)	total: 22.3s	remaining: 4.43s
1252:	test: 0.8505794	best: 0.8505794 (1252)	total: 22.4s	remaining: 4.41s
1253:	test: 0.8505830	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.39s
1254:	test: 0.8505804	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.37s
1255:	test: 0.8505794	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.35s
1256:	test: 0.8505757	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.33s
1257:	test: 0.8505742	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.31s
1258:	test: 0.8505623	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.29s
1259:	test: 0.8505431	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.27s
1260:	test: 0.8505437	best: 0.8505830 (1253)	total: 22.4s	remaining: 4.25s
1261:	test: 0.8505452	best: 0.8505830 (1253)	total: 22.5s	remaining: 4.24s
1262:	test: 0.8505359	best: 0.8505830 (1253)	total: 22.5s	remaining: 4.21s
1263:	test: 0.8506089	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.2s
1264:	test: 0.8505949	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.18s
1265:	test: 0.8506016	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.16s
1266:	test: 0.8505763	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.14s
1267:	test: 0.8505706	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.12s
1268:	test: 0.8505690	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.1s
1269:	test: 0.8505659	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.08s
1270:	test: 0.8505592	best: 0.8506089 (1263)	total: 22.5s	remaining: 4.06s
1271:	test: 0.8505799	best: 0.8506089 (1263)	total: 22.6s	remaining: 4.04s
1272:	test: 0.8505820	best: 0.8506089 (1263)	total: 22.6s	remaining: 4.02s
1273:	test: 0.8505820	best: 0.8506089 (1263)	total: 22.6s	remaining: 4.01s
1274:	test: 0.8505928	best: 0.8506089 (1263)	total: 22.6s	remaining: 3.99s
1275:	test: 0.8506187	best: 0.8506187 (1275)	total: 22.6s	remaining: 3.97s
1276:	test: 0.8506208	best: 0.8506208 (1276)	total: 22.6s	remaining: 3.95s
1277:	test: 0.8506130	best: 0.8506208 (1276)	total: 22.6s	remaining: 3.93s
1278:	test: 0.8506053	best: 0.8506208 (1276)	total: 22.6s	remaining: 3.91s
1279:	test: 0.8506068	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.89s
1280:	test: 0.8506011	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.87s
1281:	test: 0.8506068	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.85s
1282:	test: 0.8506037	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.84s
1283:	test: 0.8506037	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.82s
1284:	test: 0.8506115	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.8s
1285:	test: 0.8506115	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.78s
1286:	test: 0.8506115	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.76s
1287:	test: 0.8506182	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.74s
1288:	test: 0.8506078	best: 0.8506208 (1276)	total: 22.7s	remaining: 3.72s
1289:	test: 0.8506011	best: 0.8506208 (1276)	total: 22.8s	remaining: 3.71s
1290:	test: 0.8506172	best: 0.8506208 (1276)	total: 22.8s	remaining: 3.69s
1291:	test: 0.8506234	best: 0.8506234 (1291)	total: 22.8s	remaining: 3.67s
1292:	test: 0.8506234	best: 0.8506234 (1291)	total: 22.8s	remaining: 3.65s
1293:	test: 0.8506016	best: 0.8506234 (1291)	total: 22.8s	remaining: 3.63s
1294:	test: 0.8505990	best: 0.8506234 (1291)	total: 22.8s	remaining: 3.61s
1295:	test: 0.8506006	best: 0.8506234 (1291)	total: 22.8s	remaining: 3.6s
1296:	test: 0.8505985	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.58s
1297:	test: 0.8506078	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.56s
1298:	test: 0.8506047	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.54s
1299:	test: 0.8505980	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.52s
1300:	test: 0.8505726	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.5s
1301:	test: 0.8505949	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.48s
1302:	test: 0.8505933	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.46s
1303:	test: 0.8505928	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.45s
1304:	test: 0.8505778	best: 0.8506234 (1291)	total: 22.9s	remaining: 3.43s
1305:	test: 0.8505783	best: 0.8506234 (1291)	total: 23s	remaining: 3.41s
1306:	test: 0.8505783	best: 0.8506234 (1291)	total: 23s	remaining: 3.39s
1307:	test: 0.8506053	best: 0.8506234 (1291)	total: 23s	remaining: 3.37s
1308:	test: 0.8505768	best: 0.8506234 (1291)	total: 23s	remaining: 3.35s
1309:	test: 0.8505732	best: 0.8506234 (1291)	total: 23s	remaining: 3.33s
1310:	test: 0.8506229	best: 0.8506234 (1291)	total: 23s	remaining: 3.32s
1311:	test: 0.8506291	best: 0.8506291 (1311)	total: 23s	remaining: 3.3s
1312:	test: 0.8506166	best: 0.8506291 (1311)	total: 23s	remaining: 3.28s
1313:	test: 0.8506213	best: 0.8506291 (1311)	total: 23s	remaining: 3.26s
1314:	test: 0.8506177	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.24s
1315:	test: 0.8506223	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.23s
1316:	test: 0.8506229	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.21s
1317:	test: 0.8506151	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.19s
1318:	test: 0.8506203	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.17s
1319:	test: 0.8506197	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.15s
1320:	test: 0.8506249	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.13s
1321:	test: 0.8506182	best: 0.8506291 (1311)	total: 23.1s	remaining: 3.11s
1322:	test: 0.8506151	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.1s
1323:	test: 0.8506223	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.08s
1324:	test: 0.8506156	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.06s
1325:	test: 0.8506099	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.04s
1326:	test: 0.8506016	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.02s
1327:	test: 0.8506047	best: 0.8506291 (1311)	total: 23.2s	remaining: 3.01s
1328:	test: 0.8506027	best: 0.8506291 (1311)	total: 23.2s	remaining: 2.99s
1329:	test: 0.8505990	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.97s
1330:	test: 0.8505975	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.95s
1331:	test: 0.8505975	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.94s
1332:	test: 0.8506001	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.92s
1333:	test: 0.8505996	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.9s
1334:	test: 0.8505892	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.88s
1335:	test: 0.8505949	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.86s
1336:	test: 0.8506016	best: 0.8506291 (1311)	total: 23.3s	remaining: 2.85s
1337:	test: 0.8506643	best: 0.8506643 (1337)	total: 23.4s	remaining: 2.83s
1338:	test: 0.8506575	best: 0.8506643 (1337)	total: 23.4s	remaining: 2.81s
1339:	test: 0.8506601	best: 0.8506643 (1337)	total: 23.4s	remaining: 2.79s
1340:	test: 0.8506632	best: 0.8506643 (1337)	total: 23.4s	remaining: 2.77s
1341:	test: 0.8506694	best: 0.8506694 (1341)	total: 23.4s	remaining: 2.75s
1342:	test: 0.8506622	best: 0.8506694 (1341)	total: 23.4s	remaining: 2.74s
1343:	test: 0.8506689	best: 0.8506694 (1341)	total: 23.4s	remaining: 2.72s
1344:	test: 0.8506648	best: 0.8506694 (1341)	total: 23.4s	remaining: 2.7s
1345:	test: 0.8506658	best: 0.8506694 (1341)	total: 23.4s	remaining: 2.68s
1346:	test: 0.8506819	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.66s
1347:	test: 0.8506513	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.65s
1348:	test: 0.8506394	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.63s
1349:	test: 0.8506130	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.61s
1350:	test: 0.8506203	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.59s
1351:	test: 0.8506187	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.57s
1352:	test: 0.8506322	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.56s
1353:	test: 0.8506368	best: 0.8506819 (1346)	total: 23.5s	remaining: 2.54s
1354:	test: 0.8506135	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.52s
1355:	test: 0.8506115	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.5s
1356:	test: 0.8505861	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.48s
1357:	test: 0.8505773	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.47s
1358:	test: 0.8505747	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.45s
1359:	test: 0.8505866	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.43s
1360:	test: 0.8505871	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.41s
1361:	test: 0.8505954	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.39s
1362:	test: 0.8505908	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.38s
1363:	test: 0.8505933	best: 0.8506819 (1346)	total: 23.6s	remaining: 2.36s
1364:	test: 0.8505908	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.34s
1365:	test: 0.8505763	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.32s
1366:	test: 0.8505768	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.3s
1367:	test: 0.8505825	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.29s
1368:	test: 0.8505757	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.27s
1369:	test: 0.8505752	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.25s
1370:	test: 0.8506141	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.23s
1371:	test: 0.8506332	best: 0.8506819 (1346)	total: 23.7s	remaining: 2.21s
1372:	test: 0.8506291	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.2s
1373:	test: 0.8506146	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.18s
1374:	test: 0.8506156	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.16s
1375:	test: 0.8506099	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.14s
1376:	test: 0.8505887	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.13s
1377:	test: 0.8505887	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.11s
1378:	test: 0.8505877	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.09s
1379:	test: 0.8506151	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.07s
1380:	test: 0.8506151	best: 0.8506819 (1346)	total: 23.8s	remaining: 2.05s
1381:	test: 0.8506125	best: 0.8506819 (1346)	total: 23.9s	remaining: 2.04s
1382:	test: 0.8506203	best: 0.8506819 (1346)	total: 23.9s	remaining: 2.02s
1383:	test: 0.8506151	best: 0.8506819 (1346)	total: 23.9s	remaining: 2s
1384:	test: 0.8506275	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.98s
1385:	test: 0.8506187	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.97s
1386:	test: 0.8506146	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.95s
1387:	test: 0.8506053	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.93s
1388:	test: 0.8506125	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.91s
1389:	test: 0.8505990	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.89s
1390:	test: 0.8505913	best: 0.8506819 (1346)	total: 23.9s	remaining: 1.88s
1391:	test: 0.8505809	best: 0.8506819 (1346)	total: 24s	remaining: 1.86s
1392:	test: 0.8505902	best: 0.8506819 (1346)	total: 24s	remaining: 1.84s
1393:	test: 0.8505908	best: 0.8506819 (1346)	total: 24s	remaining: 1.82s
1394:	test: 0.8505913	best: 0.8506819 (1346)	total: 24s	remaining: 1.8s
1395:	test: 0.8505902	best: 0.8506819 (1346)	total: 24s	remaining: 1.79s
1396:	test: 0.8505944	best: 0.8506819 (1346)	total: 24s	remaining: 1.77s
1397:	test: 0.8505913	best: 0.8506819 (1346)	total: 24s	remaining: 1.75s
1398:	test: 0.8505799	best: 0.8506819 (1346)	total: 24s	remaining: 1.73s
1399:	test: 0.8505550	best: 0.8506819 (1346)	total: 24s	remaining: 1.72s
1400:	test: 0.8505587	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.7s
1401:	test: 0.8505323	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.68s
1402:	test: 0.8505276	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.67s
1403:	test: 0.8505292	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.65s
1404:	test: 0.8505219	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.63s
1405:	test: 0.8505188	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.61s
1406:	test: 0.8505286	best: 0.8506819 (1346)	total: 24.1s	remaining: 1.59s
1407:	test: 0.8505219	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.58s
1408:	test: 0.8505364	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.56s
1409:	test: 0.8505276	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.54s
1410:	test: 0.8505416	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.52s
1411:	test: 0.8505380	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.51s
1412:	test: 0.8505390	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.49s
1413:	test: 0.8505380	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.47s
1414:	test: 0.8505576	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.46s
1415:	test: 0.8505732	best: 0.8506819 (1346)	total: 24.2s	remaining: 1.44s
1416:	test: 0.8505763	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.42s
1417:	test: 0.8505701	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.4s
1418:	test: 0.8505794	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.39s
1419:	test: 0.8505726	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.37s
1420:	test: 0.8505851	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.35s
1421:	test: 0.8505778	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.33s
1422:	test: 0.8505644	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.31s
1423:	test: 0.8505669	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.3s
1424:	test: 0.8505488	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.28s
1425:	test: 0.8505178	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.26s
1426:	test: 0.8505188	best: 0.8506819 (1346)	total: 24.3s	remaining: 1.25s
1427:	test: 0.8505152	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.23s
1428:	test: 0.8505193	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.21s
1429:	test: 0.8505100	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.19s
1430:	test: 0.8505121	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.18s
1431:	test: 0.8505131	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.16s
1432:	test: 0.8505136	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.14s
1433:	test: 0.8505219	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.12s
1434:	test: 0.8505157	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.11s
1435:	test: 0.8505131	best: 0.8506819 (1346)	total: 24.4s	remaining: 1.09s
1436:	test: 0.8504924	best: 0.8506819 (1346)	total: 24.5s	remaining: 1.07s
1437:	test: 0.8505002	best: 0.8506819 (1346)	total: 24.5s	remaining: 1.05s
1438:	test: 0.8504981	best: 0.8506819 (1346)	total: 24.5s	remaining: 1.04s
1439:	test: 0.8504717	best: 0.8506819 (1346)	total: 24.5s	remaining: 1.02s
1440:	test: 0.8504929	best: 0.8506819 (1346)	total: 24.5s	remaining: 1s
1441:	test: 0.8504945	best: 0.8506819 (1346)	total: 24.5s	remaining: 986ms
1442:	test: 0.8505053	best: 0.8506819 (1346)	total: 24.5s	remaining: 969ms
1443:	test: 0.8505167	best: 0.8506819 (1346)	total: 24.5s	remaining: 952ms
1444:	test: 0.8505147	best: 0.8506819 (1346)	total: 24.5s	remaining: 934ms
1445:	test: 0.8505105	best: 0.8506819 (1346)	total: 24.6s	remaining: 917ms
1446:	test: 0.8505214	best: 0.8506819 (1346)	total: 24.6s	remaining: 900ms
1447:	test: 0.8505281	best: 0.8506819 (1346)	total: 24.6s	remaining: 883ms
1448:	test: 0.8505328	best: 0.8506819 (1346)	total: 24.6s	remaining: 866ms
1449:	test: 0.8505281	best: 0.8506819 (1346)	total: 24.6s	remaining: 849ms
1450:	test: 0.8505173	best: 0.8506819 (1346)	total: 24.6s	remaining: 832ms
1451:	test: 0.8505131	best: 0.8506819 (1346)	total: 24.6s	remaining: 815ms
1452:	test: 0.8505038	best: 0.8506819 (1346)	total: 24.7s	remaining: 798ms
1453:	test: 0.8505105	best: 0.8506819 (1346)	total: 24.7s	remaining: 780ms
1454:	test: 0.8505048	best: 0.8506819 (1346)	total: 24.7s	remaining: 763ms
1455:	test: 0.8505012	best: 0.8506819 (1346)	total: 24.7s	remaining: 747ms
1456:	test: 0.8504934	best: 0.8506819 (1346)	total: 24.7s	remaining: 730ms
1457:	test: 0.8504934	best: 0.8506819 (1346)	total: 24.7s	remaining: 713ms
1458:	test: 0.8504909	best: 0.8506819 (1346)	total: 24.8s	remaining: 696ms
1459:	test: 0.8504997	best: 0.8506819 (1346)	total: 24.8s	remaining: 679ms
1460:	test: 0.8504960	best: 0.8506819 (1346)	total: 24.8s	remaining: 662ms
1461:	test: 0.8504795	best: 0.8506819 (1346)	total: 24.8s	remaining: 645ms
1462:	test: 0.8504826	best: 0.8506819 (1346)	total: 24.8s	remaining: 628ms
1463:	test: 0.8504292	best: 0.8506819 (1346)	total: 24.9s	remaining: 611ms
1464:	test: 0.8504060	best: 0.8506819 (1346)	total: 24.9s	remaining: 594ms
1465:	test: 0.8503946	best: 0.8506819 (1346)	total: 24.9s	remaining: 577ms
1466:	test: 0.8504023	best: 0.8506819 (1346)	total: 24.9s	remaining: 561ms
1467:	test: 0.8503982	best: 0.8506819 (1346)	total: 25s	remaining: 544ms
1468:	test: 0.8504349	best: 0.8506819 (1346)	total: 25s	remaining: 527ms
1469:	test: 0.8504292	best: 0.8506819 (1346)	total: 25s	remaining: 510ms
1470:	test: 0.8504313	best: 0.8506819 (1346)	total: 25s	remaining: 493ms
1471:	test: 0.8504303	best: 0.8506819 (1346)	total: 25s	remaining: 476ms
1472:	test: 0.8504401	best: 0.8506819 (1346)	total: 25s	remaining: 459ms
1473:	test: 0.8504453	best: 0.8506819 (1346)	total: 25s	remaining: 442ms
1474:	test: 0.8504391	best: 0.8506819 (1346)	total: 25s	remaining: 425ms
1475:	test: 0.8504484	best: 0.8506819 (1346)	total: 25.1s	remaining: 407ms
1476:	test: 0.8504577	best: 0.8506819 (1346)	total: 25.1s	remaining: 390ms
1477:	test: 0.8504437	best: 0.8506819 (1346)	total: 25.1s	remaining: 373ms
1478:	test: 0.8504500	best: 0.8506819 (1346)	total: 25.1s	remaining: 357ms
1479:	test: 0.8504329	best: 0.8506819 (1346)	total: 25.1s	remaining: 340ms
1480:	test: 0.8504468	best: 0.8506819 (1346)	total: 25.2s	remaining: 323ms
1481:	test: 0.8504556	best: 0.8506819 (1346)	total: 25.2s	remaining: 306ms
1482:	test: 0.8504779	best: 0.8506819 (1346)	total: 25.2s	remaining: 289ms
1483:	test: 0.8504619	best: 0.8506819 (1346)	total: 25.2s	remaining: 272ms
1484:	test: 0.8504515	best: 0.8506819 (1346)	total: 25.2s	remaining: 255ms
1485:	test: 0.8504437	best: 0.8506819 (1346)	total: 25.2s	remaining: 238ms
1486:	test: 0.8504546	best: 0.8506819 (1346)	total: 25.2s	remaining: 221ms
1487:	test: 0.8504531	best: 0.8506819 (1346)	total: 25.2s	remaining: 204ms
1488:	test: 0.8504412	best: 0.8506819 (1346)	total: 25.2s	remaining: 187ms
1489:	test: 0.8504660	best: 0.8506819 (1346)	total: 25.3s	remaining: 170ms
1490:	test: 0.8504655	best: 0.8506819 (1346)	total: 25.3s	remaining: 153ms
1491:	test: 0.8504733	best: 0.8506819 (1346)	total: 25.3s	remaining: 136ms
1492:	test: 0.8504707	best: 0.8506819 (1346)	total: 25.3s	remaining: 119ms
1493:	test: 0.8504634	best: 0.8506819 (1346)	total: 25.3s	remaining: 102ms
1494:	test: 0.8504422	best: 0.8506819 (1346)	total: 25.3s	remaining: 84.7ms
1495:	test: 0.8504500	best: 0.8506819 (1346)	total: 25.3s	remaining: 67.7ms
1496:	test: 0.8504536	best: 0.8506819 (1346)	total: 25.3s	remaining: 50.8ms
1497:	test: 0.8504085	best: 0.8506819 (1346)	total: 25.4s	remaining: 33.9ms
1498:	test: 0.8504080	best: 0.8506819 (1346)	total: 25.4s	remaining: 16.9ms
1499:	test: 0.8504003	best: 0.8506819 (1346)	total: 25.4s	remaining: 0us

bestTest = 0.8506818695
bestIteration = 1346

Shrink model to first 1347 iterations.
Out[123]:
<catboost.core.CatBoostClassifier at 0x12068febb70>
In [131]:
test = pd.read_csv("F:\\IBPA_IIM_FinalProject\\Test_targets.csv")
pred = model.predict_proba(test.iloc[:,0])
---------------------------------------------------------------------------
CatBoostError                             Traceback (most recent call last)
<ipython-input-131-c3eae00d41c9> in <module>
      1 test = pd.read_csv("F:\\IBPA_IIM_FinalProject\\Test_targets.csv")
----> 2 pred = model.predict_proba(test.iloc[:,0])

C:\Anaconda\lib\site-packages\catboost\core.py in predict_proba(self, data, ntree_start, ntree_end, thread_count, verbose)
   4388                 with probability for every class for each object.
   4389         """
-> 4390         return self._predict(data, 'Probability', ntree_start, ntree_end, thread_count, verbose, 'predict_proba')
   4391 
   4392 

C:\Anaconda\lib\site-packages\catboost\core.py in _predict(self, data, prediction_type, ntree_start, ntree_end, thread_count, verbose, parent_method_name)
   1974         if verbose is None:
   1975             verbose = False
-> 1976         data, data_is_single_object = self._process_predict_input_data(data, parent_method_name, thread_count)
   1977         self._validate_prediction_type(prediction_type)
   1978 

C:\Anaconda\lib\site-packages\catboost\core.py in _process_predict_input_data(self, data, parent_method_name, thread_count, label)
   1960                 text_features=self._get_text_feature_indices() if not isinstance(data, FeaturesData) else None,
   1961                 embedding_features=self._get_embedding_feature_indices() if not isinstance(data, FeaturesData) else None,
-> 1962                 thread_count=thread_count
   1963             )
   1964         return data, is_single_object

C:\Anaconda\lib\site-packages\catboost\core.py in __init__(self, data, label, cat_features, text_features, embedding_features, column_description, pairs, delimiter, has_header, ignore_csv_quoting, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, feature_names, thread_count)
    453                     )
    454 
--> 455                 self._init(data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, feature_names, thread_count)
    456         super(Pool, self).__init__()
    457 

C:\Anaconda\lib\site-packages\catboost\core.py in _init(self, data, label, cat_features, text_features, embedding_features, pairs, weight, group_id, group_weight, subgroup_id, pairs_weight, baseline, feature_names, thread_count)
    924             cat_features = _get_features_indices(cat_features, feature_names)
    925             self._check_string_feature_type(cat_features, 'cat_features')
--> 926             self._check_string_feature_value(cat_features, features_count, 'cat_features')
    927         if text_features is not None:
    928             text_features = _get_features_indices(text_features, feature_names)

C:\Anaconda\lib\site-packages\catboost\core.py in _check_string_feature_value(self, features, features_count, features_name)
    495                 raise CatBoostError("Invalid {}[{}] = {} value type={}: must be int().".format(features_name, indx, feature, type(feature)))
    496             if feature >= features_count:
--> 497                 raise CatBoostError("Invalid {}[{}] = {} value: index must be < {}.".format(features_name, indx, feature, features_count))
    498 
    499     def _check_pairs_type(self, pairs):

CatBoostError: Invalid cat_features[1] = 1 value: index must be < 1.
In [ ]: